Context

StreamHive is evaluating two binary classification models to predict 30-day subscriber churn so the CRM team can send retention offers. A logistic regression model is currently in production, and a gradient-boosted tree model was trained as a challenger.

Current Performance

The Problem

The complex model performs better on offline metrics, but it is slower, harder to explain, and more expensive to maintain. The retention team can contact at most 120,000 users per week, and unnecessary offers reduce margin.

Requirements

1. Compare the simple and complex models using the metrics above and explain which differences are practically meaningful.
2. Determine whether the complex model should replace the simple model, and under what conditions.
3. Discuss how threshold choice, calibration, and business costs should affect the decision.
4. Identify what additional validation or error analysis you would run before deployment.
5. Recommend a final approach, including whether to keep the simple model, deploy the complex model, or use a hybrid strategy.

Constraints

• Churn rate is 18%.
• Retention offer cost: $4 per contacted user.
• Estimated value of preventing one churn: $38.
• Marketing operations require p95 scoring latency under 120 ms.