Choose Switchback for Marketplace Dispatch

Scenario

You work on a two-sided marketplace where the dispatch algorithm matches demand to nearby supply in real time. Your team has built a new dispatch policy that is expected to improve completed orders by reducing assignment latency, but you are worried that standard user-level A/B randomization could contaminate results because drivers and customers interact in the same shared market. You need to decide whether to run a standard A/B test or a switchback test, and design the experiment so the result is credible enough to ship.

Constraints

• Eligible traffic: 120,000 completed-or-cancelled order requests per day across 12 cities
• Average baseline completion rate: 68%; baseline median assignment latency: 42 seconds
• Maximum experiment runtime: 21 days
• Marketplace health guardrails: driver hourly earnings cannot drop by more than 2%, and cancellation rate cannot rise by more than 1 percentage point
• Engineering can support either 50/50 user-level randomization or city-by-time switchbacks in 30-minute blocks, but not both simultaneously

Question

How would you decide between a switchback test and a standard A/B test here, and how would you design, power, and analyze the experiment so that the ship decision is valid despite interference risk?