Context

StreamWave built a binary classification model to predict whether a subscriber will churn in the next 30 days so the retention team can send offers to high-risk users. The team reports that the model's overall accuracy looks strong, but campaign ROI is disappointing because many churners are still being missed.

Current Performance

Validation set size is 50,000 users. At the current threshold, the confusion matrix is: TP = 2,160, FP = 1,560, FN = 3,840, TN = 42,440.

The Problem

The model appears good if judged only by accuracy, but it may be underperforming for the actual business goal: identifying churners early enough for intervention. The retention team can contact at most 5,000 users per month, and each outreach costs $8. Saving a customer generates about $120 in expected gross margin.

Requirements

1. Interpret the model's performance using the provided metrics, not accuracy alone.
2. Explain what the confusion matrix implies for business impact.
3. Identify the main weaknesses of the current model and likely causes.
4. Recommend how you would validate the model more robustly before deployment.
5. Propose specific changes to improve performance or threshold selection.

Constraints

• Outreach capacity is capped at 5,000 users per month.
• False negatives are costly because missed churners are lost revenue.
• False positives waste retention budget and may annoy healthy subscribers.