Context

BB&T has deployed a binary classification model in its consumer lending workflow to predict whether a new personal loan applicant will default within 12 months. The model is used in pre-approval screening inside the BB&T underwriting process, where false negatives increase credit losses and false positives reduce booked loan volume.

Current Performance

The Problem

The Head of Credit Risk is concerned that the model now catches a smaller share of actual defaulters even though precision improved. BB&T wants to understand whether the issue is threshold choice, class imbalance, calibration drift, or a broader shift in applicant behavior.

Requirements

1. Interpret what the metric changes imply about model behavior in production.
2. Explain why accuracy alone is not sufficient for this lending use case.
3. Use the confusion matrix and business costs to assess whether the current threshold is appropriate.
4. Identify likely sources of performance degradation and propose a validation plan.
5. Recommend specific model or monitoring improvements for the next release.

Constraints

• BB&T must keep adverse action rates explainable and compliant.
• Manual underwriting capacity is limited to 3,500 additional reviews per month.
• Average loss from a missed defaulter is $4,800; average opportunity cost of incorrectly declining a good applicant is $620.