StreamBox uses a binary classification model to predict which subscribers are likely to churn in the next 30 days so the retention team can send discount offers. A recently deployed logistic regression model looks strong on overall accuracy, but the retention team says too many churners are still being missed.
| Metric | Validation Set | Previous Model |
|---|---|---|
| Accuracy | 0.91 | 0.87 |
| Precision | 0.62 | 0.54 |
| Recall | 0.38 | 0.57 |
| F1 Score | 0.47 | 0.55 |
| AUC-ROC | 0.79 | 0.76 |
| Churn Rate | 0.12 | 0.12 |
The VP of Growth is questioning whether the new model is actually better, since higher accuracy has not translated into better retention targeting. You need to evaluate the model properly and explain what the metrics imply for business performance.