
You are analyzing an A/B test for a new mobile gameplay feature in a live game. The outcome metric is noisy because player activity varies a lot day to day, and you want to improve sensitivity without changing the product experience or rerunning the test.
How would you use CUPED or another variance-reduction approach in this experiment analysis? Explain when it is appropriate, how you would implement it, and what checks you would run to make sure the estimated treatment effect remains valid.
Noisy engagement metricPotential sample ratio mismatchPossible unit mismatch between sessions and playersNeed to preserve unbiased treatment effect estimation