Context

StreamBox uses a binary classification model to predict which subscribers are likely to cancel in the next 30 days so the retention team can send targeted offers. A recently deployed logistic regression model looks strong on overall accuracy, but churn has continued to rise and the CRM team says too many at-risk users are being missed.

Current Performance

The Problem

The new model improved accuracy and precision, but recall dropped sharply. As a result, many customers who actually churn are not being targeted by retention campaigns. The Head of Growth wants to know whether this model is truly better and what changes should be made before the next monthly campaign.

Requirements

1. Interpret the current metrics and explain why accuracy alone is misleading here.
2. Compare the new model against the previous model using the business objective of churn reduction.
3. Use the confusion matrix implications to explain the cost of false positives vs. false negatives.
4. Recommend how you would validate the model more thoroughly before rollout.
5. Propose specific improvements to increase business impact, not just headline metrics.

Constraints

• Retention budget allows outreach to at most 10,000 customers per month.
• A false positive costs about $2 in incentive spend.
• A false negative costs an estimated $45 in lost contribution margin.