Metrics for Experiments

HardA/B Testing & Experimentation00:00

Practice interviewer

In session

5 left

00:00

Your question is Metrics for Experiments. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

You need to log in / sign up to chat or submit.

Problem

What metrics would you use?

Asked in the tech round, during the A/B testing and statistics stage. This is a follow-up on choosing primary, secondary, and guardrail metrics for an experiment.

Explain how you would:

Select and define a single primary metric, including its unit of analysis and business rationale.
Choose secondary metrics without allowing exploratory results to drive the launch decision.
Define guardrails and actionable thresholds.
Set an MDE, calculate the required sample size, and explain how you would analyze the result.
State the ship, do-not-ship, and iterate rules, including how you would handle peeking and sample ratio mismatch.