Your question is Pitfalls in Mobile Game Experiments. Take a moment with it on the right.
Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).
You work on a mobile game and are running experiments on a new feature, offer, or progression change. Early results can look promising, but mobile game behavior is noisy and treatment effects can be distorted by player interactions, short-term excitement, or execution issues in the rollout.
What are the main pitfalls you watch for when running experiments on a mobile game? How would you design and analyze the test so you can trust the result before shipping?