Choose Metrics for Churn Model | Dataford Interview Questions

Your question is Choose Metrics for Churn Model. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

Context

StreamBox has deployed a binary classification model to predict whether a subscriber will churn in the next 30 days. The product team wants to use the model to trigger retention offers, but leadership is concerned that the team is reporting only accuracy while the business cost of missed churners is much higher than the cost of sending an unnecessary offer.

Current Performance

Metric	Current Model	Previous Model	Change
Accuracy	0.91	0.89	+0.02
Precision	0.58	0.54	+0.04
Recall	0.36	0.49	-0.13
F1 Score	0.44	0.51	-0.07
AUC-ROC	0.79	0.77	+0.02
Log Loss	0.29	0.33	-0.04
Churn rate in evaluation set	0.12	0.12	0.00

The Problem

The current model looks better on accuracy and AUC-ROC, but it is catching fewer actual churners. The retention team has budget to contact only 25,000 users per month, so the company needs to decide which metrics should drive model selection and thresholding.

Requirements

Explain which metrics you would use to evaluate this churn model and why.
Interpret the current results relative to the previous model and identify the main tradeoff.
Use the provided metrics and confusion matrix to assess whether the current model is suitable for deployment.
Recommend how you would choose a decision threshold given the retention budget.
Suggest concrete next steps to improve evaluation and model performance.

Constraints

Monthly retention budget covers at most 25,000 offers.
Each unnecessary offer costs $4.
Each missed churner is estimated to cost $65 in lost gross margin.
Churn labels are finalized 30 days after prediction, so feedback is delayed.