Detect Concept Drift in Credit Risk

Context

LendWise uses a gradient-boosted binary classifier to approve or decline personal loan applications. The model was stable at launch, but over the last quarter loan defaults increased even though offline monitoring shows only a modest drop in aggregate discrimination metrics.

Current Performance

The Problem

The risk team suspects concept drift after LendWise expanded into gig-worker and thin-file customer segments. You need to determine whether the issue is threshold drift, calibration drift, feature distribution shift, or a true change in the relationship between features and default risk.

Requirements

1. Interpret the metric changes and explain what type of drift is most likely occurring.
2. Propose a monitoring framework to detect concept drift in production before defaults materially rise.
3. Identify what analyses you would run by segment, time window, and score band.
4. Recommend specific mitigation steps for the next 30 days and the next 2 quarters.
5. Explain how you would validate that the mitigation improved business outcomes, not just offline metrics.

Constraints

• Loan decisions must remain fully automated for 95% of applications.
• Retraining requires 3 weeks because labels mature after 90 days.
• A 1-point increase in approval rate is worth about $420K quarterly revenue, but each additional default costs $1,900 on average.