A/B Testing Approach

HardA/B Testing & Experimentation00:00

Practice interviewer

In session

5 left

00:00

Your question is A/B Testing Approach. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

You need to log in / sign up to chat or submit.

Problem

How would you test a feature with A/B testing?

Asked in the culture fit stage. Treat this as a practical design exercise and make your assumptions explicit rather than waiting for a predefined product context.

State the null and alternative hypotheses, primary metric, MDE, and guardrails.
Choose the randomization unit and explain the allocation and duration.
Show a sample-size calculation using clearly labeled illustrative assumptions.
Describe the pre-registered analysis plan, including peeking, multiple comparisons, and SRM checks.
Give a ship, do-not-ship, or iterate rule and identify important risks.