Your question is Handle Peeking in Customer Experiments. Take a moment with it on the right.
Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).
You work on a customer product team running an A/B test on a new experience. The team has been checking results repeatedly during the test, and there are concerns that multiple comparisons across metrics or segments may be inflating false positives.
How would you handle peeking or multiple comparisons when analyzing the experiment, and what would you pre-register so the final decision is statistically sound?