LendWise uses a binary classification model to predict whether a small-business loan applicant will default within 12 months. The current gradient boosting model was deployed to support underwriting decisions, but business leaders are concerned that overall accuracy looks strong while default losses remain high.
| Metric | Validation Set | Production Last 60 Days | Baseline Rules Engine |
|---|---|---|---|
| Accuracy | 0.91 | 0.90 | 0.84 |
| Precision | 0.68 | 0.61 | 0.42 |
| Recall | 0.74 | 0.49 | 0.57 |
| F1 Score | 0.71 | 0.54 | 0.48 |
| AUC-ROC | 0.87 | 0.81 | 0.69 |
| Default Rate | 9.0% | 11.8% | 11.8% |
| Monthly charge-off loss | $1.9M | $3.1M | $3.8M |
The model still appears better than the old rules engine, but recall has fallen sharply in production. As a result, many risky applicants are being approved, and finance estimates that missed defaults are driving most of the recent increase in charge-off losses.