Please suggest some simple functions that I can use to test my neural network implementation

1

I am a beginner and I have developed code in Octave to train a neural network (as part of Andrew Ng's coursera course).

Now I would like to generate some datasets in Excel for simple mathematical functions and try and train my neural network implementation.

I tried y = x^2 and y = sin(x), but my training accuracy is only around 67%.

I have varied the number of hidden layer units, varied the lambda, but still don't get good results. Where could I be wrong?

Vin

Posted 2016-03-14T16:16:25.640

Reputation: 89

Answers

1

Without code you could have made to many mistakes to answer your question directly.

However, I suggest two things:

  1. Gradient checking (e.g. this explanation, but there are many more - pretty sure also in the course)
  2. XOR problem with a 2:2:1 network with only sigmoid activations. Plot the error surface. Compare the decision surface with that of this interactive demo.

It should look similar to this: enter image description here

However, please keep in mind that the problem might not be your implementation, but rather the network architecture / hyperparameters such as the number of epochs you're training or the training data.

Also, very important: I doubt that you will get good results for $y = x^2$, except if $x$ is restricted to $[-1,1]$ or something similar simple. Please keep the domain of your output layer in mind.

Martin Thoma

Posted 2016-03-14T16:16:25.640

Reputation: 15 590

Martin could you clarify why restricting your domain is necessary in training the network? – mburke05 – 2016-05-18T23:04:58.700

One the one hand you have to have a look at the range of values of the activation function of the last layer. Now you might simply take a linear output, but then the last weights might get enormous. Even small fluctuations at previous layers might so have a huge impact on the last one and this on the output – Martin Thoma – 2016-05-18T23:10:48.347