Context

LendWise uses a binary classification model to predict whether a small-business loan applicant will default within 12 months. The current gradient boosting model was deployed to support underwriting decisions, but business leaders are concerned that overall accuracy looks strong while default losses remain high.

Current Performance

The Problem

The model still appears better than the old rules engine, but recall has fallen sharply in production. As a result, many risky applicants are being approved, and finance estimates that missed defaults are driving most of the recent increase in charge-off losses.

Requirements

1. Explain which metrics you would prioritize for this use case and why accuracy alone is misleading.
2. Interpret the production metrics and identify the most likely failure modes.
3. Use the confusion matrix and segment-level errors to diagnose what the model is missing.
4. Recommend specific evaluation methods to validate whether this is thresholding, calibration, or data drift.
5. Propose concrete model and process improvements, including how you would measure success after changes.

Constraints

• False negatives are expensive: each missed default costs about $18,000 on average.
• False positives reduce loan approvals and hurt revenue and customer experience.
• Underwriting can manually review only 1,200 borderline applications per month.
• Any model update must be explainable enough for risk and compliance review.