Business Context

Didi Chuxing is considering a new in-app coupon banner on the Didi Rider home screen to increase first ride booking among newly registered users. The Growth team wants a statistically sound A/B test design before launch.

Problem Statement

Design the experiment and evaluate whether the proposed sample size is sufficient. Then, using pilot results, determine whether the treatment meaningfully improves conversion.

Given Data

Baseline first-ride booking conversion for new users over 7 days is estimated at 18.0%. Product wants to detect at least a 1.5 percentage point absolute lift. Traffic is split 50/50 between control and treatment for 14 days.

Requirements

1. Define the null and alternative hypotheses for the primary metric.
2. Compute the required sample size per group for a two-sided two-proportion test.
3. Assess whether the planned 18,000 users per group is adequate.
4. Using the pilot results, calculate the observed treatment lift, z-statistic, p-value, and a 95% confidence interval.
5. State whether Didi should roll out the coupon banner based on both statistical and practical significance.
6. Name at least two guardrail metrics you would monitor during the experiment.

Assumptions

• User-level randomization is valid and each new user appears once.
• No major traffic allocation bugs or sample ratio mismatch.
• The normal approximation for proportions is appropriate at this sample size.
• The primary metric is 7-day first-ride booking conversion.