StreamWave uses a binary classification model to predict which subscribers are likely to cancel in the next 30 days so the retention team can send targeted offers. The current model is a logistic regression classifier, but leadership believes it is underperforming on the customers who matter most: true churn risks.
| Metric | Validation Set | Previous Model |
|---|---|---|
| Accuracy | 0.91 | 0.89 |
| Precision | 0.62 | 0.55 |
| Recall | 0.38 | 0.51 |
| F1 Score | 0.47 | 0.53 |
| AUC-ROC | 0.79 | 0.76 |
| Log Loss | 0.31 | 0.36 |
| Predicted Positive Rate | 0.08 | 0.13 |
| Actual Churn Rate | 0.12 | 0.12 |
Although the new model improved accuracy and precision, recall fell from 0.51 to 0.38. As a result, many churners are not being flagged for intervention. The retention team can only contact 25,000 users per week, so the company needs a model that performs well within outreach capacity.