Context

LendWise uses a binary classification model to predict whether a personal loan applicant will default within 12 months. The current gradient boosted tree model was deployed 4 months ago to support underwriting decisions, but business stakeholders are concerned that overall accuracy looks acceptable while default losses remain higher than expected.

Current Performance

In production over the last 30 days, the model evaluated 50,000 applications. At the current threshold, it produced: TP = 4,305, FP = 2,751, FN = 6,195, TN = 36,749.

The Problem

The Head of Risk asks: “What steps would you take to improve this model’s accuracy?” A strong answer should go beyond simply saying “collect more data” and should diagnose whether accuracy is even the right optimization target given the current error profile.

Requirements

1. Interpret the model’s current performance using the provided metrics.
2. Explain why accuracy alone may be misleading in this setting.
3. Identify likely causes of the production drop versus validation.
4. Recommend concrete steps to improve model performance, including evaluation and validation changes.
5. Discuss whether threshold changes, feature work, retraining, or data quality fixes should be prioritized.

Constraints

• False negatives are costly because missed defaulters create direct credit losses.
• False positives reduce approval volume and hurt revenue.
• The risk team can only retrain the model once per month.