Problem
Scenario
You are reviewing an A/B test for a product change intended to improve a key conversion outcome. The team agrees the experiment needs one main success metric, but there is debate about which other metrics should be monitored so the change does not improve one outcome while harming the overall user experience.
Question
What is the difference between a primary metric and a guardrail metric in an experiment? How would you explain the role each one plays in deciding whether to ship a change?
Metric Setup
MDE·5% relative lift on conversionGuardrails·Bounce rate, error rate, 7-day retentionPrimary metric·Conversion rate
You are practicing as a guest. Sign up free to get your answer graded with AI feedback. Your draft stays right here.
