Experiment Duration for Signup Lift

Business Context

StreamCart is testing a new signup page and wants to know how long the A/B test must run before it can reliably detect the expected improvement. The primary metric is visitor-to-signup conversion.

Problem Statement

The team expects a 2% relative lift over a 10% baseline conversion rate, so the treatment conversion rate is expected to be 10.2%. Estimate the required sample size per variant and convert that into experiment duration.

Given Data

Requirements

1. Convert the relative lift into an absolute effect size.
2. Compute the required sample size per group for a two-sample test of proportions.
3. Estimate the total sample size needed across both groups.
4. Convert the sample requirement into the number of days the experiment should run.
5. Briefly explain whether the answer changes if the team wants to capture full weekly seasonality.

Assumptions

• Random assignment at the user level with no interference between users.
• Two-sided hypothesis test with constant traffic over time.
• Normal approximation for two proportions is appropriate at this scale.
• Ignore attrition, sample ratio mismatch, and metric definition changes during the test.