LendWise uses a binary classification model to predict whether a personal loan applicant will default within 90 days. A recently deployed gradient boosting model was expected to reduce credit losses, but post-launch performance is below target and the risk team is questioning whether the model should remain in production.
| Metric | Validation Before Launch | Current Production Holdout | Target |
|---|---|---|---|
| Accuracy | 0.84 | 0.78 | 0.82 |
| Precision | 0.69 | 0.58 | 0.65 |
| Recall | 0.74 | 0.49 | 0.70 |
| F1 Score | 0.71 | 0.53 | 0.67 |
| AUC-ROC | 0.81 | 0.73 | 0.80 |
| Log Loss | 0.44 | 0.58 | <0.48 |
| Default rate | 12.0% | 14.8% | — |
| Approval rate | 61% | 66% | 60-63% |
The model is approving more applicants than expected while missing a larger share of true defaulters. Credit losses rose from $1.9M to $2.7M per month over the last quarter, and leadership wants a structured plan to diagnose whether the issue is thresholding, drift, feature quality, or model generalization.