Different accuracy for different rng values

2

While tuning the SVM classification model in Matlab, I came across the rng function in matlab in which seed (stabilizes the random shuffling of the data in the algorithm) is changed. When the function called is rng(1) then I am getting one accuracy value (99%). When it is changed to rng(2) then I am getting another value (57%). So there is a huge change in accuracy as visible. What does this mean? Am I training it wrong?

The train and test set correct rate (in %) that I am getting with different runs without changing rng are(train,test)
(96, 82.8)
(94.6, 95.3)
(96, 85.9)
(96, 90)
(95, 95)

girl101

Posted 2016-08-09T05:45:59.030

Reputation: 1 093

1What are the testing errors of both? Nothing can be said by just looking at the training errors. Maybe rng(1) is better than rng(2) or maybe it is overfitting. – Dawny33 – 2016-08-09T05:48:31.843

@Dawny33 So does this mean that there is overfitting? A good model should have the same accuracy for any rng right? – girl101 – 2016-08-09T05:52:22.610

Answers

0

The training errors in this dataset has a huge difference (99% vs 57%). So, maybe the one with the rng(1) split has overfitted your dataset.

So there is a huge change in accuracy as visible. What does this mean? Am I training it wrong?

The huge change might be due to overfitting. (Also, judge the model through validation curves, and then fit a model which balances the bias-variance plot.)

Dawny33

Posted 2016-08-09T05:45:59.030

Reputation: 7 606

made a small change in the question. Let me know whether still there is over-fitting or not – girl101 – 2016-08-09T07:15:28.183

1@Rishika No, I don't think there's any overfitting there. But, that's without changing the rng value, right? – Dawny33 – 2016-08-09T07:17:17.180

yes. This is without changing rng – girl101 – 2016-08-09T07:22:02.310

1@Rishika The model looks good to me, then :) – Dawny33 – 2016-08-09T07:25:52.940