Context

StreamWave built a binary classification model to predict whether a subscriber will churn in the next 30 days so the retention team can send discounts and outreach. The team reports that the model's overall accuracy looks strong, but campaign ROI is disappointing and many churners are still being missed.

Current Performance

The Problem

The new gradient boosted tree model appears better on accuracy and AUC-ROC, yet it identifies far fewer true churners than the previous model. Leadership wants to know which metrics should matter most here, whether the current model is actually better, and what changes should be made before the next retention campaign.

Requirements

1. Explain which classification metrics you would prioritize for this use case and why.
2. Interpret the current metrics together rather than in isolation.
3. Diagnose what the confusion matrix suggests about model behavior.
4. Recommend how to choose or tune the decision threshold.
5. Propose concrete next steps to improve business performance.

Constraints

• Monthly retention budget covers offers for at most 8,000 customers.
• A false negative costs about $120 in lost margin.
• A false positive costs about $8 in unnecessary incentive spend.
• The model score must be explainable enough for marketing operations.