Problem
Scenario
You work on a consumer fintech product and an A/B test on a new in-app prompt is showing an apparent improvement in conversion. The team is encouraged by the early lift, but you want to know whether the result reflects a true effect or random variation.
Question
What would you look for to decide whether the observed lift is real rather than noise? Walk through how you would validate the result before recommending a launch decision.
What to Examine
- Effect size and 95% confidence interval
- P-value from the pre-registered primary test
- Whether the experiment reached planned sample size
- Stability over time and across key segments
- Guardrail metric health and SRM checks
Practicing as: Data Scientist interview at World Wide TechnologyHi, I'll play your World Wide Technology interviewer for the Data Scientist role. Candidates describe these interviews as mostly positive and moderately difficult, so expect me to be friendly and conversational. Take your time with the question above and answer like we're in the room.
You are practicing as a guest. Sign up free to get your answer graded with AI feedback. Your draft stays right here.
