Dataford
Interview QuestionsInterview GuidesExperiencesMock InterviewsPricing
Get started

Primary vs Guardrail Metric Choice

MediumA/B Testing & Experimentation00:00
Practice interviewer
In session
5 left
00:00

Your question is Primary vs Guardrail Metric Choice. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

You need to log in / sign up to chat or submit.

Problem

Scenario

You are planning an A/B test for a mobile game feature and need to decide how success will be measured. The feature may improve short-term engagement, but there is concern it could also hurt longer-term player experience or monetization in ways that would be missed if you optimize for only one outcome.

Question

How do you decide which metric should be the primary metric and which metrics should be guardrails in this experiment? Walk through how you would make that choice, how it affects experiment design, and how you would interpret the results if the primary metric improves but a guardrail worsens.

Metric Setup

MDE·2% relative liftUnit·player_idGuardrails·ARPDAU, crash-free session rate, D7 retentionPrimary metric·D1 session-start rate