Minimum Detectable Effect for Signup Test | Dataford Interview Questions

Your question is Minimum Detectable Effect for Signup Test. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

Business Context

StreamCart is planning an A/B test on a new signup page. Before launch, the product team wants to know the smallest conversion lift the experiment can reliably detect with the traffic available over a 3-week test window.

Problem Statement

Determine the minimum detectable effect (MDE) for a two-sample test of proportions, assuming a two-sided hypothesis test, 5% significance level, and 80% power. Then convert that MDE into the minimum detectable number of additional signups.

Given Data

Metric	Value
Baseline signup conversion rate	8.4%
Expected eligible visitors during test	240,000
Traffic split	50% control / 50% treatment
Significance level	0.05
Desired power	0.80
Test type	Two-sided

This means each variant will receive 120,000 visitors if the test runs as planned.

Requirements

State the hypothesis-testing setup used for MDE planning.
Compute the per-group sample size.
Use the normal approximation for two independent proportions to calculate the absolute MDE in percentage points.
Convert the absolute MDE into a relative lift versus baseline.
Estimate the minimum additional signups needed in treatment for the effect to be detectable at the planned sample size.
Briefly explain how MDE would change if traffic were lower or if the team wanted 90% power instead of 80%.

Assumptions

Random assignment is valid and independent across users.
Signup is a binary outcome measured once per visitor.
Baseline conversion is stable during the test window.
Use the standard planning approximation with equal-sized groups and variance evaluated at the baseline rate.