2

0

I have tried to train a neural network for a simple x^2 function

- I developed training data in excel. First column (X) is =RANDBETWEEN(-5,5) i.e random integer between -5 and 5
- Second column simply squares first column
- And third column, my output 'y' column is 0 or 1. 0 if second column is less than 12.5 else 1

I made 850 training examples and used the first column as 'X' and third column as 'y'

However **I am only able to get a training accuracy of 63%!**

Where could I have gone wrong? I changed input_layer to 1 and tried hidden units between 5 and 35. Tried regularization lambda 0 to 2 but still only 63% accuracy! Where could I have gone wrong?

My predict function is p = 1 if h2(i)>0.5 else 0.

Any help will be much appreciated! :-)

**I also noticed that my neural network's output is 0.3XXX for all training examples**...how is this possible??

What is the architecture of your neural network? How many layers, what type of activation's, number of nodes, etc... – Armen Aghajanyan – 2016-03-14T15:03:30.207

I used input layer of 1 unit, one hidden layer of 15 units (tried up to 25 units) and output layer of 1 unit. For activation I used the sigmoid function. – Vin – 2016-03-14T15:43:00.570

Is the sigmoid activation function applied to the output layer as well? – Armen Aghajanyan – 2016-03-15T00:20:33.047

Yes for the output layer as well – Vin – 2016-03-15T00:45:45.770

Thanks Armen. I will try without the sigmoid today evening. However in my training data set the output y is not linear, I have converted y to 0 or 1 based on whether the linear output is greater than or less than 12.5; don't you think it should work for such a case with sigmoid function? – Vin – 2016-03-15T02:10:03.867

1Can you post the code somewhere? Have you scaled the input data to -1, 1? How did you initialize the weights and what learning rates did you try? Can you plot the learning curves - if they don't decrease, learning rate might be too low, if they jump around a lot, learning rate might be too high. You should definitely use the sigmoid function also on the output, don't remove it. – stmax – 2016-03-15T07:59:12.837

I'm voting to close this question as off-topic because we generally close questions as not useful to future readers if they were ultimately due to a typo or other local error – Sean Owen – 2016-03-18T13:20:58.333