I used the Neural Network Toolbox in matlab to train my data. I used four training algorithms, Scaled Conjugate Gradient (SCG), Gradient Descent with momentum and adaptive learning back-propagation (GDX), Resilient Back Propagation (RBP) and Broyden-Fletcher-Goldfarb-Shanno quasi-Newton back propagation (BFG). I have fixed the seeds at different points and obtained the accuracy. This is what I get:
The first column contains the size of the feature set. I have added features and increased the size of feature set to analyse the performance.
Initially I have ranked the features and then taken the top 8 feature as one set, the top 16 feature as the next set and so on. The first number before the '-' , is the performance of the algorithm on the training set, the second number after the '-' is the accuracy of the testing set. The train and test set has been divided into 60 and 20 respectively. The other 20 is the validation set. The learning algorithms each have been run with the same seed values to fix the accuracy.
I have fixed the seed to obtain each of the results btw, Like I have used rng(1), rng(10), rng(158), rng(250) and averaged the results to obtain one single pair of train-test accuracy, and I have done this for each pairs.
As you can see I am getting the same accuracy for all feature set size for each of the individual training algorithm. The same data shows perturbation in SVM chen I change the set size. What does this mean?