LendWise uses a binary classification model to predict whether a personal loan applicant will default within 12 months. The team reports strong overall accuracy, but the credit risk manager is concerned that too many risky borrowers are still being approved.
| Metric | Validation Set | Baseline Rule Model | Change |
|---|---|---|---|
| Accuracy | 0.91 | 0.84 | +0.07 |
| Precision | 0.68 | 0.52 | +0.16 |
| Recall | 0.42 | 0.61 | -0.19 |
| F1 Score | 0.52 | 0.56 | -0.04 |
| AUC-ROC | 0.79 | 0.71 | +0.08 |
| Default rate in data | 0.12 | 0.12 | — |
Confusion matrix on 10,000 validation applicants:
| Predicted Default | Predicted Non-Default | |
|---|---|---|
| Actual Default | 504 | 696 |
| Actual Non-Default | 237 | 8,563 |
The model looks good if you focus on accuracy alone, but missed defaults are expensive because they lead to loan losses. The team needs a better evaluation of whether this model is actually suitable for deployment and what should be improved.