Context
Lyft is considering a change to the rider home screen promo module that makes a limited-time ride discount more prominent during the trip-request flow. The Growth team wants to know whether the change increases completed rides enough to justify rollout.
Hypothesis Seed
The proposed treatment replaces the current compact promo card with a larger, personalized discount banner on the Lyft rider app home screen. Product believes this will increase the share of eligible riders who complete at least one ride within 7 days, but it could also reduce contribution margin if discounts are overused.
Constraints
- Eligible traffic: 1.2M rider app home-screen visitors per day in the U.S.
- About 35% of visitors are eligible for the promo experiment
- Maximum experiment window: 21 days, with a decision needed before the next pricing calendar update
- False positives are costly because they can create ongoing promo spend and margin loss
- False negatives are also meaningful because rider frequency is a core growth lever, but the business prefers to be conservative on shipping
- Assume baseline 7-day ride conversion among eligible riders is 18.0%
- Assume the smallest business-relevant lift is +3% relative in 7-day ride conversion
Deliverables
- Define the null and alternative hypotheses, the primary metric, and 2-4 guardrail metrics for this Lyft experiment.
- Estimate the minimum detectable effect and compute the required sample size per arm using explicit assumptions for alpha, power, and baseline conversion.
- Choose the unit of randomization, allocation, duration, and any stratification or variance-reduction approach you would use.
- Pre-register an analysis plan: statistical test, peeking policy, multiple-comparison policy, and how you will handle any mismatch between unit of randomization and unit of analysis.
- State a clear ship / don’t ship / iterate rule that respects guardrails, and identify key pitfalls such as novelty effects, sample ratio mismatch, and interference across riders or markets.
