Context

LendWise built a binary classification model to predict whether a personal loan applicant will default within 12 months. The model is scheduled for deployment to support underwriting decisions, but the risk team wants a clear validation plan before it goes live.

A holdout test set of 20,000 recent applications was evaluated after training on 180,000 historical applications. Default prevalence in the test set is 10%, and false negatives are more costly than false positives because missed defaulters create direct credit losses.

Current Performance

The Problem

The model looks strong on accuracy, but recall drops materially on the unseen test set. The underwriting team is concerned that the model may be overestimating readiness for production and missing too many risky borrowers.

Requirements

1. Explain how you would validate this model before deployment beyond checking a single test-set score.
2. Interpret what the gap between validation and test performance suggests.
3. Identify which metrics matter most for this use case and why.
4. Recommend how you would use threshold tuning, calibration, and error analysis before launch.
5. Propose a deployment validation plan, including monitoring after release.

Constraints

• Each false negative costs about $4,500 in expected credit loss.
• Each false positive rejects a potentially good customer and costs about $300 in lost margin.
• The model must remain explainable enough for risk and compliance review.