Context

StreamWave uses a binary classification model to predict which subscribers are likely to cancel in the next 30 days so the retention team can send targeted offers. The current model is a logistic regression classifier, but leadership believes it is underperforming on the customers who matter most: true churn risks.

Current Performance

The Problem

Although the new model improved accuracy and precision, recall fell from 0.51 to 0.38. As a result, many churners are not being flagged for intervention. The retention team can only contact 25,000 users per week, so the company needs a model that performs well within outreach capacity.

Requirements

1. Interpret the current metrics and explain whether this model is actually better than the previous one.
2. Diagnose what the metric pattern suggests about threshold choice, class imbalance, and business fit.
3. Recommend how you would refine the model to improve overall performance.
4. Propose a validation strategy to confirm the refinements work before deployment.
5. Explain which metric(s) should drive decision-making for this use case.

Constraints

• Weekly retention outreach capacity is fixed at 25,000 users.
• A false negative costs about $48 in lost gross margin.
• A false positive costs about $4 in discount and contact expense.
• Retraining can happen weekly, but feature engineering resources are limited.