Rippling wants to test a new onboarding prompt in the Rippling app switcher that encourages newly invited admins to complete payroll setup. Before launching, the Product Growth team wants to size the experiment so it has enough power to detect a meaningful lift without running unnecessarily long.
Use the baseline conversion rate and target effect size to determine the required sample size per variant for a two-arm A/B test on a binary conversion metric. Then estimate how long the test will need to run given expected traffic, and assess whether the observed post-launch result would be statistically significant.
| Metric | Value |
|---|---|
| Baseline payroll setup completion rate | 18.0% |
| Minimum detectable absolute lift | 1.5 percentage points |
| Significance level | 0.05 |
| Desired power | 80% |
| Traffic eligible for experiment per day | 24,000 admins |
| Traffic split | 50% control / 50% treatment |
| Observed control sample after launch | 41,200 |
| Observed treatment sample after launch | 41,000 |
| Observed control conversions | 7,416 |
| Observed treatment conversions | 8,036 |
Assume a two-sided test and independent Bernoulli outcomes.