StreamCart is testing a new signup page intended to increase free-trial starts. The product manager wants to know how long the experiment must run before the team can reasonably expect to detect a meaningful lift with statistical significance.
Estimate the required sample size and runtime for a two-arm A/B test on conversion rate, then check whether the currently observed data are already statistically significant.
| Metric | Value |
|---|---|
| Baseline signup conversion rate | 8.2% |
| Minimum detectable effect (relative lift) | 10.0% |
| Significance level | 5.0% |
| Desired power | 80.0% |
| Daily eligible users | 52,000 |
| Traffic split | 50% control / 50% treatment |
| Current runtime | 10 days |
| Control users so far | 260,000 |
| Treatment users so far | 260,000 |
| Control conversions so far | 21,320 |
| Treatment conversions so far | 23,140 |
The business defines success as detecting at least a 10% relative lift over the baseline conversion rate.