Context

Tiger Analytics has deployed a binary classification model in a client retention workflow to predict whether a subscription customer will churn in the next 30 days. The model is used inside a Tiger Analytics decisioning dashboard to prioritize outreach by the retention team, but business leaders are concerned that the model looks strong on headline metrics while still missing too many high-risk customers.

Current Performance

Confusion matrix on 50,000 customers at the current threshold:

The Problem

The current model has better accuracy, precision, AUC-ROC, and log loss than the previous version, but recall has fallen materially. The retention team can contact only 4,500 customers per week, and each missed churner has a much higher business cost than an unnecessary outreach.

Requirements

1. Interpret the model’s performance using the metrics above rather than relying on accuracy alone.
2. Explain what the confusion matrix implies for business impact.
3. Diagnose why this model may be underperforming operationally despite improved headline metrics.
4. Recommend how you would evaluate threshold changes, validation strategy, and segment-level performance.
5. Propose specific improvements to increase business value.

Constraints

• Positive class is imbalanced at 12% churn.
• Retention team capacity is capped at 4,500 outreach actions per week.
• False negatives are more costly than false positives.
• Any recommendation must be measurable in Tiger Analytics production monitoring.