Your question is Considerations for A/B Test Design. Take a moment with it on the right.
Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).
What considerations would you take into account when designing an A/B test for a new feature?
Discuss how you would define the hypothesis, select a primary metric and guardrails, choose the randomization unit, and determine the sample size and test duration. Explain how you would pre-register the analysis, handle peeking and multiple comparisons, and assess risks such as novelty effects, network interference, SUTVA violations, Simpson's paradox, and sample ratio mismatch.
Be prepared to state a practical ship or do-not-ship rule and explain how you would distinguish statistical significance from meaningful product impact.