Problem
Scenario
You work on a collaborative work-management product and want to evaluate a change to a shared surface, such as task notifications, inbox ranking, or workload recommendations. A standard user-level A/B test may be hard to run cleanly because users interact inside teams and projects, exposure can spill across collaborators, or the rollout may need to happen by workspace or over time.
Question
How would you decide whether to use a quasi-experiment instead of a randomized test in this situation? Walk through the conditions that would push you away from randomization, what quasi-experimental design you would choose, and how you would judge whether the evidence is strong enough to act on.
What This Tests
- Recognizing when randomization is infeasible versus merely inconvenient
- Handling interference in collaborative products
- Choosing between cluster randomization, switchbacks, and quasi-experimental designs
- Defining metrics, power, and guardrails even when the final design is non-randomized
Practicing as: Data Scientist interview at UberHi, I'll play your Uber interviewer for the Data Scientist role. Candidates describe these interviews as mixed and moderately difficult, so expect me to be professional and fair. Take your time with the question above and answer like we're in the room.
You are practicing as a guest. Sign up free to get your answer graded with AI feedback. Your draft stays right here.


