Validate Credit Risk Model Reliability

Context

LendWise uses a gradient boosting classifier to approve or decline unsecured personal loan applications. The model performed well in offline validation, but after 8 weeks in production the risk team observed that default rates are higher than expected in some applicant segments, raising concerns about real-world reliability.

Current Performance

The Problem

The model appears to be underestimating risk in production and missing a meaningful share of future defaulters. Leadership wants a validation plan that goes beyond a single holdout score and demonstrates whether the model is reliable enough for continued deployment.

Requirements

1. Diagnose what the production metrics suggest about real-world reliability.
2. Explain which additional validation checks you would run before trusting the model.
3. Identify likely root causes for the offline-to-production gap.
4. Recommend how to validate calibration, threshold choice, and segment-level stability.
5. Propose concrete actions to improve deployment confidence.

Constraints

• False negatives (approving risky borrowers) are much more costly than false positives.
• Loan decisions must remain explainable for compliance review.
• Retraining can occur only once per month due to governance and audit requirements.