Validate Loan Default Classifier

Context

LendWise built a binary classification model to predict whether a personal loan applicant will default within 12 months. The team wants to deploy the model to support underwriting decisions, but offline results look uneven across metrics and leadership wants a clear validation plan before launch.

Current Performance

The model was trained on 180,000 historical applications and evaluated on a 20,000-row holdout set with a 10% default rate.

The Problem

At first glance, 91% accuracy looks strong, but the risk team is concerned the model may be missing too many actual defaulters. The business needs to decide whether this model is ready for deployment, whether the threshold should change, and what additional validation steps are required before using predictions in production.

Requirements

1. Interpret the current metrics and explain whether the model is actually performing well.
2. Identify which validation methods should be used before deployment.
3. Analyze the confusion matrix and explain the business cost of false positives vs. false negatives.
4. Recommend whether the decision threshold should remain at 0.50.
5. Propose specific steps to improve confidence in the model before launch.

Constraints

• False negatives are costly because missed defaulters increase credit losses.
• False positives are also costly because good applicants may be rejected unnecessarily.
• The underwriting team requires a model that is explainable and stable across monthly applicant cohorts.