LendWise uses a binary classification model to predict whether a personal loan applicant will default within 12 months. The model was recently deployed to support underwriting decisions, but the credit risk team is concerned that headline accuracy looks strong while business outcomes remain mixed.
| Metric | Validation Set | Previous Baseline | Change |
|---|---|---|---|
| Accuracy | 0.91 | 0.88 | +0.03 |
| Precision | 0.62 | 0.55 | +0.07 |
| Recall | 0.38 | 0.52 | -0.14 |
| F1 Score | 0.47 | 0.53 | -0.06 |
| AUC-ROC | 0.79 | 0.76 | +0.03 |
| Default Rate | 0.12 | 0.12 | 0.00 |
Additional validation-set counts (20,000 applicants): TP = 912, FP = 558, FN = 1,488, TN = 17,042.
The model appears better on accuracy and precision, but it is missing many actual defaulters. The underwriting team wants to know whether the model is truly effective and whether the current decision threshold is appropriate.