Design an Experiment | Dataford Interview Questions

Your question is Design an Experiment. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

How would you design an experiment?

Asked in the tech round, A/B testing and statistics stage. This was part of the technical case study on experimentation and metrics.

Explain how you would turn the question into a valid online experiment and make a ship or do-not-ship decision. Your response should address:

The hypothesis, primary metric, guardrails, and minimum detectable effect.
The unit of randomization, allocation, duration, and traffic requirements.
Sample-size and power calculations using explicit assumptions.
The statistical analysis plan, including peeking and multiple comparisons.
Common validity risks such as sample ratio mismatch, novelty, interference, and SUTVA violations.

Your question is Design an Experiment. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

How would you design an experiment?

Asked in the tech round, A/B testing and statistics stage. This was part of the technical case study on experimentation and metrics.

Explain how you would turn the question into a valid online experiment and make a ship or do-not-ship decision. Your response should address:

The hypothesis, primary metric, guardrails, and minimum detectable effect.
The unit of randomization, allocation, duration, and traffic requirements.
Sample-size and power calculations using explicit assumptions.
The statistical analysis plan, including peeking and multiple comparisons.
Common validity risks such as sample ratio mismatch, novelty, interference, and SUTVA violations.