LendWise uses a binary classification model to predict whether a personal loan applicant will default within 90 days. The current model was recently deployed to support underwriting decisions, but business stakeholders are concerned that headline accuracy looks strong while default losses remain higher than expected.
| Metric | Validation Set | Previous Model | Change |
|---|---|---|---|
| Accuracy | 0.94 | 0.91 | +0.03 |
| Precision | 0.68 | 0.61 | +0.07 |
| Recall | 0.42 | 0.57 | -0.15 |
| F1 Score | 0.52 | 0.59 | -0.07 |
| AUC-ROC | 0.81 | 0.78 | +0.03 |
| Default rate in data | 0.08 | 0.08 | 0.00 |
| Applicants flagged high-risk | 4.9% | 7.5% | -2.6 pts |
The underwriting team sees fewer applicants flagged as high-risk, but post-loan analysis shows too many actual defaulters are still being approved. You need to assess what to look at when evaluating this model beyond a single metric and explain whether the current model is actually better for the business.