You ran an experiment on a booking flow change. The headline result is a 5% increase in bookings, but cancellations also rose by 2%. You need to decide whether the treatment should ship.
An A/B test shows a 5% increase in bookings but a 2% increase in cancellations. Do we launch?
Primary metric versus guardrail metric reasoningHypothesis testing for two proportionsConfidence intervals, not just point estimatesNet impact on completed trips and business value