Validate Loan Default Model Analysis

Context

LendWise uses a binary classification model to predict whether a small-business loan applicant will default within 90 days. A junior analyst prepared a performance review showing that the latest model is "better than baseline," but the credit risk director wants the analysis validated before it is presented to executives.

Current Performance

The Problem

The current model appears stronger on accuracy, precision, and AUC-ROC, but recall has fallen materially. Since missed defaulters are costly, leadership is concerned the analysis may overstate model quality if it relies on a narrow set of metrics or an unvalidated validation process.

Task

1. Explain how you would validate the analyst's evaluation before presenting it.
2. Identify which checks you would run on the data split, label quality, and metric calculations.
3. Interpret whether the current model is actually better for the business.
4. Recommend what additional analyses or validation views should be included.
5. State whether you would keep the current threshold of 0.50 or revisit it.

Constraints

• Each false negative default costs about $18,000 in expected loss.
• Each false positive rejection costs about $1,200 in lost contribution margin.
• The credit team requires a transparent recommendation within 48 hours.
• Retraining is possible, but only after the evaluation is validated.