Business Context

CartJet is a large e-commerce marketplace (~8M weekly active users) optimizing its mobile checkout. A redesign reduces the number of form fields and adds Apple Pay earlier in the flow. Finance estimates that each additional completed order is worth $6.40 in contribution margin on average, so even small conversion lifts are meaningful.

The team wants to run a controlled A/B test (50/50 traffic split) and needs to decide how long to run the experiment to reliably detect a meaningful improvement.

Problem Statement

You are asked to determine the minimum sample size per group required to detect a lift in checkout conversion rate with adequate power, and then translate that into estimated test duration given expected traffic.

Given Data

Requirements

1. State the null and alternative hypotheses for the primary metric (conversion rate).
2. Compute the minimum required sample size per group using a normal approximation for a two-sample test of proportions.
3. Convert the sample size into an estimated test duration in days given the daily eligible traffic and 50/50 split.
4. Briefly explain what inputs most strongly drive sample size (and how product decisions like choosing a smaller MDE affect runtime).

Assumptions and Constraints

• Users are randomly assigned and each user contributes at most one checkout attempt (independent Bernoulli outcomes).
• Use a two-sided hypothesis test at α = 0.05.
• Use the common planning approximation for two proportions:
  • Use p̄ = (p1 + p2)/2 in the variance term (planning value).
  • z-values from the standard normal distribution.
• Ignore sequential testing/peeking adjustments (assume you will not stop early based on interim looks).