Business Context

Chime is testing four new in-app prompts in the Chime app to increase activation of a first qualifying direct deposit. Traffic is split evenly across one control and four treatment variants, and the Product Growth Analyst needs to decide which variants, if any, should move forward while controlling false positives.

Problem Statement

You are evaluating several variants at once, so a naive approach that compares each treatment to control at $\alpha = 0.05$ inflates the family-wise error rate. Use a multiple-testing correction to determine which variants remain statistically significant.

Given Data

Primary metric: proportion of new members who set up a qualifying direct deposit within 14 days.

Use a two-sided test for each treatment vs. control. Overall family-wise significance level should be controlled at $\alpha = 0.05$. Assume independent random assignment and no sample ratio mismatch.

Requirements

1. Compute the unadjusted two-proportion z-test statistic and p-value for each variant vs. control.
2. Apply a Bonferroni correction across the 4 comparisons.
3. State the adjusted significance threshold.
4. Determine which variants remain significant after correction.
5. Briefly explain why multiple-testing correction is needed here and one alternative to Bonferroni.

Assumptions

• Randomization was user-level and balanced across arms.
• The normal approximation is valid because each arm has large sample size.
• The business wants to control family-wise error rate rather than only false discovery rate.