Assess Performance Drop in Customer Churn Prediction Model

Context

ChurnGuard, a SaaS company, utilizes a logistic regression model to predict customer churn for its 500,000 subscribers. Recently, the marketing team reported a decline in the model's effectiveness in identifying at-risk customers, prompting a review of its performance metrics.

Current Performance

The Problem

Despite stable precision, the drop in recall indicates the model is failing to identify a significant portion of customers likely to churn. This has resulted in a 50% increase in monthly revenue loss due to churn, raising concerns among stakeholders.

Requirements

1. Diagnose the reasons behind the recall drop from 78% to 65%.
2. Identify potential data drift or feature relevance issues.
3. Propose at least three hypotheses for the performance decline.
4. Recommend specific model improvements and their expected impact.
5. Discuss the implications of adjusting the churn prediction threshold.

Constraints

• Model retraining must occur within a 2-week cycle.
• The marketing budget for customer retention campaigns is limited to $500,000 monthly.