A/B Test for Chime Sign-Up Flow

Business Context

Chime wants to test a redesigned sign-up flow on its landing page to increase completed account applications without hurting downstream quality. A 14-day randomized experiment was run on new visitors to Chime's landing page.

Problem Statement

Evaluate whether the new sign-up flow improves the primary conversion metric enough to justify rollout, and quantify the uncertainty around the lift.

Given Data

Additional experiment parameters:

Requirements

1. State the null and alternative hypotheses for the sign-up conversion rate.
2. Compute the observed conversion rates and absolute lift.
3. Run a two-proportion z-test using the pooled standard error.
4. Calculate the two-sided p-value.
5. Construct a 95% confidence interval for the difference in conversion rates.
6. Determine whether the result is statistically significant at the 5% level.
7. Briefly assess whether the observed lift is practically meaningful for Chime.
8. Using the planning assumptions, estimate the required sample size per variant for 80% power.

Assumptions

• Visitors were randomly assigned at the user level and counted once.
• No major instrumentation issues or sample-ratio mismatch occurred.
• The normal approximation is valid because both groups have large sample sizes.
• The primary metric is completed sign-up conversion on the landing-page flow.