Monitor Deployed Loan Model Drift

Context

LendWise uses a binary classification model to approve or decline small-business loan applications. The model was deployed 6 months ago, and the risk team now suspects that post-deployment performance has degraded even though approval volume has increased.

Current Performance

The Problem

The business wants a monitoring framework that can detect performance degradation early, explain whether the issue is threshold drift, calibration drift, or population shift, and define when retraining or rollback is required.

Requirements

1. Explain what these metric changes suggest about model performance over time.
2. Propose a production monitoring plan with leading and lagging indicators.
3. Identify likely root causes for the recall and calibration decline.
4. Recommend alert thresholds, review cadence, and retraining triggers.
5. Discuss how you would separate data drift from true concept drift.

Constraints

• Loan decisions must be returned in under 300 ms.
• True default labels arrive with a 90-day delay.
• Compliance requires stable approval policies and documented threshold changes.
• A false negative (missed defaulter) costs about $4,200; a false positive (declined good borrower) costs about $320 in lost contribution margin.