Context

StreamWave uses a binary classification model to predict whether a subscriber will churn in the next 30 days so the retention team can send discount offers. A logistic regression model was recently deployed, but leadership is concerned that the model looks strong on overall accuracy while still missing too many churners.

Current Performance

The Problem

The model is more selective than the previous baseline and generates fewer retention offers, but it is also missing a large share of customers who actually churn. The retention team wants to know whether this model is truly better and how to evaluate it beyond a single metric.

Requirements

1. Interpret what the current metrics say about model performance.
2. Explain why high accuracy may be misleading in this setting.
3. Use the confusion matrix and class imbalance to assess business impact.
4. Recommend which metrics should matter most for this use case and why.
5. Propose specific changes to improve model evaluation and model performance.

Constraints

• Monthly retention budget supports at most 10,000 offers.
• Each retention offer costs $8.
• Missing a true churner costs an estimated $95 in lost contribution margin.
• Product and marketing teams prefer simple, explainable models for campaign targeting.