Dataford
Interview QuestionsInterview GuidesExperiencesMock InterviewsPricing
Get started

When to Prefer Quasi-Experiments

HardA/B Testing & Experimentation00:00
Practice interviewer
In session
5 left
00:00

Your question is When to Prefer Quasi-Experiments. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

You need to log in / sign up to chat or submit.

Problem

Scenario

You work on a collaborative work-management product and want to evaluate a change to a shared surface, such as task notifications, inbox ranking, or workload recommendations. A standard user-level A/B test may be hard to run cleanly because users interact inside teams and projects, exposure can spill across collaborators, or the rollout may need to happen by workspace or over time.

Question

How would you decide whether to use a quasi-experiment instead of a randomized test in this situation? Walk through the conditions that would push you away from randomization, what quasi-experimental design you would choose, and how you would judge whether the evidence is strong enough to act on.

What This Tests

  • Recognizing when randomization is infeasible versus merely inconvenient
  • Handling interference in collaborative products
  • Choosing between cluster randomization, switchbacks, and quasi-experimental designs
  • Defining metrics, power, and guardrails even when the final design is non-randomized