Dataford
Interview QuestionsInterview GuidesExperiencesMock InterviewsPricing
Get started

Incrementality Test Without User Randomization

HardA/B Testing & Experimentation00:00
Practice interviewer
In session
5 left
00:00

Your question is Incrementality Test Without User Randomization. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

You need to log in / sign up to chat or submit.

Problem

Scenario

You work on a retail media team measuring whether a campaign drove incremental sales, but you cannot cleanly randomize exposure at the user level because delivery and matching are imperfect. You still need a credible experiment design that can estimate lift while accounting for spillovers and operational bias.

Question

How would you think about incrementality testing for this campaign when user-level randomization is not feasible? Walk through the experiment design you would choose, how you would measure impact, and what risks could bias the estimate.

Hypothesis & Metrics

Null·No incremental sales lift versus control geos/storesGuardrails·Net revenue/ROAS proxy, cannibalization, delivery balanceAlternative·Positive incremental sales liftPrimary metric·Incremental sales per eligible household

Design Considerations

  • User-level randomization is not feasible
  • Spillovers across markets may bias estimates
  • Observed exposure is not a valid causal comparison
  • Cluster-level assignment may be more credible