Context

NovaBank built a new gradient-boosted classifier to predict 30-day credit card default at account origination. The team wants to know whether the model has real predictive signal or is only showing weak lift over the current scorecard, especially because approvals affect both loss rates and customer growth.

Current Performance

The Problem

Leadership is asking whether these results indicate strong enough signal to replace the current scorecard and how much of the gain is due to ranking power versus threshold choice or calibration. You need to assess whether the classifier is genuinely useful, where it still fails, and what changes would improve deployment readiness.

Requirements

1. Assess the model's signal strength using the metrics above, not just one metric.
2. Explain whether the model is better at ranking, probability estimation, or both.
3. Use the confusion matrix and segment errors to identify where signal is weak.
4. Recommend thresholding and calibration changes for production.
5. Propose follow-up validation steps before launch.

Constraints

• False negatives increase charge-off losses by about $1,200 per missed default.
• False positives reject profitable applicants and cost about $180 each in lifetime margin.
• Risk policy requires predicted probabilities to be reasonably calibrated for pricing and line assignment.