Dataford
Interview QuestionsInterview GuidesExperiencesMock InterviewsPricing
Get started

Prove Feature X Moves Metric Y

MediumA/B Testing & Experimentation00:00
Practice interviewer
In session
5 left
00:00

Your question is Prove Feature X Moves Metric Y. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

You need to log in / sign up to chat or submit.

Problem

Scenario

You work on a product team that is considering a change to feature X, and the team wants to know whether it is actually affecting metric Y. You have enough traffic to run an online experiment, but the result needs to be credible enough to support a launch decision.

Question

How would you prove whether the change in feature X is affecting metric Y? Walk through how you would define the test, choose the right metric and unit of randomization, and decide whether the result is strong enough to ship.