4

1

I am trying to learn a very simple sequence using an RNN (implemented in Keras)

The input sequence is randomly generated integers between 0 to 100:

x=np.random.randint(0,100, size=2000)

while the expected output value for time t is the (t-2)th input term i.e:

y_{t}=x_{t-2}

such that an example dataset looks like this:

+------+------+ | X | Y | +------+------+ | 0 | NA | | 24 | NA | | 33 | 0 | | 6 | 24 | | 78 | 33 | | 11 | 6 | | . | . | | . | . | +------+------+

Note: I drop the NA rows before training.

I am trying to train a simple RNN to learn this sequence as below:

```
xtrain=np.reshape(df['X'], (df.shape[0], 1, 1))
#to match dimension of input shape for SimpleRNN layer.
model=Sequential()
model.SimpleRNN(2, input_shape=(None,1)))
model.add(Dense(1))
model.compile(optimizer='adam', loss='mse')
model.fit(x=xtrain, y=df['Y'], epochs=200, batch_size=5)
```

however, I find that this implementation results in a local minima which predicts constant value(~50) for all test observations.

Could anyone help me with the right way of implementing a basic RNN in Keras to learn this sequence?

Your network might seem to small for it to memorize a sequence of 100 integers. – Daniel – 2018-09-02T20:37:13.337

@Aditya, could you share the code for the solution of your problem? I'm interested too. – Basj – 2019-02-11T10:38:50.047