Business Context
StreamCart, a subscription video platform, wants to test a simplified signup page. The team does not want to launch the experiment until they know how many users are needed to reliably detect a meaningful lift in signup conversion.
Problem Statement
Determine the minimum sample size per group required for a two-arm A/B test comparing conversion rates. The team wants enough power to detect a small but commercially meaningful improvement from the current baseline.
Given Data
| Metric | Value |
|---|
| Baseline conversion rate p1 | 8.2% |
| Minimum detectable effect (absolute lift) | 1.0 percentage point |
| Expected treatment conversion rate p2 | 9.2% |
| Significance level α | 0.05 |
| Desired power $1-\beta$ | 80% |
| Test type | Two-sided |
| Traffic split | 50/50 |
Assume the product team will use a normal approximation for two independent proportions.
Requirements
- State the null and alternative hypotheses.
- Compute the critical values for α and power.
- Calculate the required sample size per variant for a two-proportion A/B test.
- Calculate the total required sample size.
- If StreamCart receives 52,000 eligible users per day and only 60% can be exposed to the experiment, estimate the test duration.
- Briefly explain how the required sample size would change if the MDE were smaller or if the team wanted 90% power instead of 80%.
Assumptions
- Users are independently randomized and counted once.
- Conversion is binary at the user level.
- No peeking or early stopping adjustments are planned.
- The baseline rate estimate of 8.2% is stable enough for planning purposes.
