Evaluate Churn Model on Imbalance

Scenario

You own a binary churn prediction model used in Salesforce Marketing Cloud to trigger retention outreach for a subscription business. The model is a gradient-boosted tree classifier scored weekly, and customers above a 0.50 threshold are sent to a limited retention budget campaign. Churn is rare, so leadership is concerned that the model's headline accuracy looks strong while the campaign is not reducing lost customers as much as expected. You are asked to evaluate whether the model is actually useful on this imbalanced dataset and whether the current threshold is appropriate.

Performance Data

Question

How would you evaluate this model given the class imbalance, and what would you recommend to decide whether to keep the current threshold, change the operating point, or use a different evaluation approach?