Problem
Scenario
You are planning an A/B test for a mobile game feature and need to decide how success will be measured. The feature may improve short-term engagement, but there is concern it could also hurt longer-term player experience or monetization in ways that would be missed if you optimize for only one outcome.
Question
How do you decide which metric should be the primary metric and which metrics should be guardrails in this experiment? Walk through how you would make that choice, how it affects experiment design, and how you would interpret the results if the primary metric improves but a guardrail worsens.
Metric Setup
MDE·2% relative liftUnit·player_idGuardrails·ARPDAU, crash-free session rate, D7 retentionPrimary metric·D1 session-start rate
You are practicing as a guest. Sign up free to get your answer graded with AI feedback. Your draft stays right here.



