2

1

Say I've built a (completely unrealistic) classification model in Keras that gives me 1.00 accuracy.

And next, I would like to use my model on some new, unseen data, and use `model.predict_proba`

to get a probability that the observation belongs to class "A". Say this returns to me a 0.75.

Am I interpreting this correctly in English: "100 percent of the time, the model is confident that this new observation is 75 percent likely to be class A" ?

If this is correct, then let's consider if my model was not totally perfect, like in real life, and instead it gave me a 0.40 accuracy. Say my `predict_proba`

is still 0.75. Then, is this correct:

"40 percent of the time, the model is confident that this new observation is 75 percent likely to be class A." ?

If so...this makes it seem like `predict_proba()`

is not tell a complete story.

I could mislead someone (say a journalist...or a judge, whoever) by saying, "There's a 75 percent chance this unseen observation belongs to class A"...and that might sound great, if I fail to reveal that this statment was based on a model that had a low accuracy like 0.40.

Am I stating this correctly, and does my apprehension have validity?

makes sense.... – Monica Heddneck – 2017-08-08T09:47:24.290