Problem
Scenario
You work on a ride-sharing marketplace and are testing a redesigned rider home screen that highlights saved places and a more prominent destination entry. The team believes the change will increase ride-request conversion by reducing friction for repeat riders, but the expected lift is small and noisy because rider intent varies a lot day to day. You want to use CUPED or another variance-reduction method to improve sensitivity without changing the user experience.
Constraints
- Eligible traffic: 1.2M rider app opens per day from 420k unique riders
- Maximum experiment duration: 14 days
- Allocation must be 50/50 after a brief safety ramp
- Booking conversion cannot decline by more than 0.5pp, and rider cancellation rate cannot increase by more than 0.3pp
Question
How would you design and analyze this experiment, including whether and how you would use CUPED or another variance-reduction technique, and what decision rule you would use to ship or not ship the new experience?
Practicing as: Product Growth Analyst interview at AircallHi, I'll play your Aircall interviewer for the Product Growth Analyst role. Candidates describe these interviews as often stressful and moderately difficult, so expect me to be direct and to the point. Take your time with the question above and answer like we're in the room.
You are practicing as a guest. Sign up free to get your answer graded with AI feedback. Your draft stays right here.
