Business Context

CartBridge is a large e-commerce marketplace (~18M monthly active users) optimizing its mobile checkout. A redesign reduces the number of form fields and introduces address auto-complete. The PM is excited because the A/B test shows a higher purchase conversion rate, but Finance and Risk want to know how big the lift could realistically be before approving a full rollout (the change also increases third‑party address-lookup costs).

You’re asked to explain and quantify the significance of confidence intervals: not just whether the effect is “statistically significant,” but what range of true effects is consistent with the data.

Problem Statement

Using the experiment results below, compute and interpret a 95% confidence interval (CI) for the difference in conversion rates (treatment − control). Then connect that CI to a hypothesis test decision and to a business rollout decision.

Given Data

Notes: The randomization unit is a user. Each user is counted once (first exposure). Assume independence between users and that the normal approximation for proportions is acceptable.

Requirements

1. Compute $\hat p_A$, $\hat p_B$, and the observed lift $\hat\Delta = \hat p_B - \hat p_A$.
2. Compute a 95% CI for $\Delta = p_B - p_A$ using the unpooled standard error.
3. Use the CI to answer: is the result statistically significant at $\alpha=0.05$ (two-sided)? Explain why the CI answers the same question as a hypothesis test.
4. Interpret the CI in business terms: give a plausible range for incremental monthly purchases if CartBridge has 18M MAUs and 35% of MAUs reach checkout each month.
5. Briefly explain why reporting only a p-value would be insufficient for the rollout decision.

Assumptions and Constraints

• Random assignment was implemented correctly (no targeting).
• No interference (one user’s assignment doesn’t affect another’s outcome).
• The test ran long enough to average out day-of-week effects.
• You are not correcting for multiple comparisons (assume this was the primary metric).