Scenario

You work on a consumer fintech product and an A/B test on a new in-app prompt is showing an apparent improvement in conversion. The team is encouraged by the early lift, but you want to know whether the result reflects a true effect or random variation.

Question

What would you look for to decide whether the observed lift is real rather than noise? Walk through how you would validate the result before recommending a launch decision.