Test Pause Orders Feature

Merchants get too many orders and stop accepting them; DoorDash introduced a feature to pause orders. Design an A/B test to measure this product.

Asked in the onsite case stage. Clarify the available traffic, test duration, merchant eligibility, and whether the feature is automatically triggered or manually controlled. Then address:

1. State the hypothesis and define a primary metric, guardrails, and an explicit MDE.
2. Choose the unit of randomization and analysis, calculate sample size, and assess whether the traffic supports the test.
3. Pre-register the statistical analysis, including peeking, multiple comparisons, and sample ratio mismatch checks.
4. Explain likely pitfalls, including novelty, interference, and mismatch between merchant-level assignment and order-level outcomes.
5. Give a ship, do-not-ship, or iterate rule that respects guardrails.