Context

StreamBox uses a binary classification model to predict whether a subscriber will churn in the next 30 days so the CRM team can send retention offers. The current model appears strong on headline accuracy, but churn has continued to rise and the marketing team believes too many at-risk users are being missed.

Current Performance

The Problem

The VP of Growth is concerned that the team is relying too heavily on accuracy even though only 12% of users churn. The model is identifying fewer users for intervention, and the business suspects the threshold is too conservative for a retention use case where missing churners is costly.

Task

1. Explain which classification metrics you would use to assess this model and why accuracy alone is insufficient.
2. Interpret the current metrics and describe what they imply about model behavior.
3. Use the confusion matrix information below to discuss the tradeoff between false positives and false negatives.
4. Recommend how StreamBox should evaluate threshold changes and which metric(s) should guide deployment.
5. Propose concrete next steps to improve model evaluation and model performance.

Constraints

• Retention team can contact at most 15,000 users per week.
• A retention offer costs $4 per contacted user.
• Missing a true churner leads to an average expected revenue loss of $38.