Dataford
Interview QuestionsInterview GuidesExperiencesMock InterviewsPricing
Get started

Diagnose Sample Ratio Mismatch

HardA/B Testing & Experimentation00:00
Practice interviewer
In session
5 left
00:00

Your question is Diagnose Sample Ratio Mismatch. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

You need to log in / sign up to chat or submit.

Experiment Readout With Allocation Imbalance

Scenario

You work on a digital product team and an A/B test has just come back with an unexpected imbalance between control and treatment traffic. The primary metric looks promising, but the allocation does not match what was planned, and there are signs of possible data quality issues in the readout.

Question

How would you handle sample ratio mismatch or other data quality issues in the experiment readout, and how would you decide whether the result is trustworthy enough to act on?