Validate New Loan Default Model

Context

LendWise uses a binary classification model to predict whether a personal loan applicant will default within 90 days. The team trained a new gradient boosting model to replace the current logistic regression scorecard, and leadership wants to know whether the new model truly improves business outcomes rather than just offline metrics.

Current Performance

The Problem

The new model catches more risky applicants but also declines more borderline customers, reducing approval volume. Product and risk leaders disagree on whether the observed lift is meaningful enough to justify rollout.

Task

1. Explain how you would test whether the new model improves outcomes versus the current approach.
2. Identify which metrics should be primary, secondary, and guardrail metrics.
3. Assess whether the metric changes suggest a better operating point or only a threshold shift.
4. Propose an offline and online validation plan, including segmentation and statistical testing.
5. Recommend whether to launch, run a champion-challenger test, or reject the model.

Constraints

• False approvals create high credit losses, but unnecessary declines reduce revenue and customer growth.
• Loan decisions must remain explainable for compliance review.
• A full production rollout affects ~120,000 applications per month.