3

I am evaluating a Credit Risk model that predicts the estimated likelihood of customers defaulting on their mortgage accounts. The model is a Logistic Regression estimator and was built by another team. They use the Gini metric to measure the performance of the model. They achieved 87%. Upon evaluation, I found that the recall was 51% whilst the error rate of the non rare event class (do not default) was 0.9%. Am I correct in thinking that the Gini is actually a misleading metric in this case because it doesn't really show the extremely poor predictive performance of the rare event class? I have questioned them about this and tried to recommend them to use precision/recall metrics as well as confusion matrices and a precision-recall trade-off graph but they quickly dismissed me.

Any advice would be much appreciated.

Nonsense. One of the main motivation of auc is to work well for data with skewed class priors and miss classification costs. – cs0815 – 2021-02-11T22:15:13.910

What exactly is nonsense? The relationship between the gini coefficient and the roc auc is a mathematical fact. If the imbalance is

severelyskewed towards the negative class, there are scenarios where the roc auc might be overly optimistic. I'm simply pointing that out as a "potential issue" not saying it always has to be. – oW_ – 2021-02-12T00:40:12.337The auc, based on the roc curve, is chosen when there are skewed class priors and miss classification costs. Thus I do not aggree with this part: "influenced by class imbalance" (sorry I should not have used the word nonsense). – cs0815 – 2021-02-12T08:27:30.203