Interview Guides
You are planning an A/B test on a growth change in the feed. The baseline click-through rate on the module you care about is 6.0%, and you want 80% power at a 5% two-sided significance level to detect a minimum relative lift of 5%. You expect a 50/50 traffic split and want to size the experiment using a normal approximation for two proportions.
How many users do you need per variant, how long would the test need to run if each variant gets about 220,000 eligible users per day, and how would you interpret the trade-off between sample size, power, and minimum detectable effect?
