Monitor Production Credit Risk Model

Context

LendFlow uses a gradient boosting classifier to predict whether a loan applicant will become 60+ days delinquent within 6 months. The model was validated offline before launch, but after 4 months in production, business teams report rising default losses and more borderline approvals.

Current Performance

The Problem

The model is still being used with the original decision threshold of 0.35, but production outcomes suggest weaker ranking power and poorer probability quality. You need to define how to measure and monitor production performance, diagnose what likely changed, and recommend actions.

Requirements

1. Interpret what the metric changes imply about production model health.
2. Identify the most likely causes of the degradation and what monitoring would detect them early.
3. Explain whether threshold, calibration, or retraining should be prioritized first.
4. Propose a production monitoring framework covering model, data, and business metrics.
5. Recommend specific next steps to reduce credit losses without sharply cutting approvals.

Constraints

• Loan decisions must be returned in under 250 ms.
• Retraining can only happen monthly due to compliance review.
• Reducing approval rate below 55% would materially hurt revenue.
• False negatives are much more expensive than false positives because missed risky borrowers create charge-offs.