Sample Size for Signup Experiment | Dataford Interview Questions

Business Context

StreamCart, a subscription video platform, wants to test a simplified signup page. The team does not want to launch the experiment until they know how many users are needed to reliably detect a meaningful lift in signup conversion.

Problem Statement

Determine the minimum sample size per group required for a two-arm A/B test comparing conversion rates. The team wants enough power to detect a small but commercially meaningful improvement from the current baseline.

Given Data

Metric	Value
Baseline conversion rate $p_1$	8.2%
Minimum detectable effect (absolute lift)	1.0 percentage point
Expected treatment conversion rate $p_2$	9.2%
Significance level $\alpha$	0.05
Desired power $1-\beta$	80%
Test type	Two-sided
Traffic split	50/50

Assume the product team will use a normal approximation for two independent proportions.

Requirements

State the null and alternative hypotheses.
Compute the critical values for $\alpha$ and power.
Calculate the required sample size per variant for a two-proportion A/B test.
Calculate the total required sample size.
If StreamCart receives 52,000 eligible users per day and only 60% can be exposed to the experiment, estimate the test duration.
Briefly explain how the required sample size would change if the MDE were smaller or if the team wanted 90% power instead of 80%.

Assumptions

Users are independently randomized and counted once.
Conversion is binary at the user level.
No peeking or early stopping adjustments are planned.
The baseline rate estimate of 8.2% is stable enough for planning purposes.

Problem

Business Context

Problem Statement

Given Data

Metric	Value
Baseline conversion rate $p_1$	8.2%
Minimum detectable effect (absolute lift)	1.0 percentage point
Expected treatment conversion rate $p_2$	9.2%
Significance level $\alpha$	0.05
Desired power $1-\beta$	80%
Test type	Two-sided
Traffic split	50/50

Assume the product team will use a normal approximation for two independent proportions.

Requirements

State the null and alternative hypotheses.
Compute the critical values for $\alpha$ and power.
Calculate the required sample size per variant for a two-proportion A/B test.
Calculate the total required sample size.
If StreamCart receives 52,000 eligible users per day and only 60% can be exposed to the experiment, estimate the test duration.
Briefly explain how the required sample size would change if the MDE were smaller or if the team wanted 90% power instead of 80%.

Assumptions

Users are independently randomized and counted once.
Conversion is binary at the user level.
No peeking or early stopping adjustments are planned.
The baseline rate estimate of 8.2% is stable enough for planning purposes.

Problem

Business Context

Problem Statement

Given Data

Metric	Value
Baseline conversion rate $p_1$	8.2%
Minimum detectable effect (absolute lift)	1.0 percentage point
Expected treatment conversion rate $p_2$	9.2%
Significance level $\alpha$	0.05
Desired power $1-\beta$	80%
Test type	Two-sided
Traffic split	50/50

Assume the product team will use a normal approximation for two independent proportions.

Requirements

State the null and alternative hypotheses.
Compute the critical values for $\alpha$ and power.
Calculate the required sample size per variant for a two-proportion A/B test.
Calculate the total required sample size.
If StreamCart receives 52,000 eligible users per day and only 60% can be exposed to the experiment, estimate the test duration.
Briefly explain how the required sample size would change if the MDE were smaller or if the team wanted 90% power instead of 80%.

Assumptions

Users are independently randomized and counted once.
Conversion is binary at the user level.
No peeking or early stopping adjustments are planned.
The baseline rate estimate of 8.2% is stable enough for planning purposes.

Problem

Business Context

Problem Statement

Given Data

Metric	Value
Baseline conversion rate $p_1$	8.2%
Minimum detectable effect (absolute lift)	1.0 percentage point
Expected treatment conversion rate $p_2$	9.2%
Significance level $\alpha$	0.05
Desired power $1-\beta$	80%
Test type	Two-sided
Traffic split	50/50

Assume the product team will use a normal approximation for two independent proportions.

Requirements

State the null and alternative hypotheses.
Compute the critical values for $\alpha$ and power.
Calculate the required sample size per variant for a two-proportion A/B test.
Calculate the total required sample size.
If StreamCart receives 52,000 eligible users per day and only 60% can be exposed to the experiment, estimate the test duration.
Briefly explain how the required sample size would change if the MDE were smaller or if the team wanted 90% power instead of 80%.

Assumptions

Users are independently randomized and counted once.
Conversion is binary at the user level.
No peeking or early stopping adjustments are planned.
The baseline rate estimate of 8.2% is stable enough for planning purposes.