You support a product team running online experiments on a travel booking surface. They need a results view that helps them understand whether a treatment is working, whether the effect is trustworthy, and whether any guardrail metrics are being harmed. The challenge is to present statistical evidence clearly without encouraging misreads from noisy early data.
How would you design a visualization to support experimentation results for a product team? Explain what you would show for the primary metric, guardrails, uncertainty, and experiment health so that teams can make better launch decisions.
Primary metric lift with confidence intervalGuardrail status and thresholdsExperiment health checks such as SRMReadiness state: underpowered vs decision-ready