
You work on a customer analytics team for a subscription-based consumer service. The business wants to try a new customer outreach strategy to improve response and downstream retention, but outreach can also annoy customers or increase opt-outs if it is too aggressive.
How would you evaluate whether the new customer outreach strategy is working? Walk through how you would design the experiment, define success, and decide whether to roll it out.
Hypothesis articulationPrimary metric and guardrailsPower analysis with explicit MDEExperiment design and randomization choiceAwareness of peeking and other pitfalls