Problem
Scenario
You work on a digital product where a new growth feature was rolled out to some users or regions for operational reasons, so treatment was not randomly assigned. Early results look promising, but you are concerned that selection effects and timing differences could bias the estimated impact.
Question
How would you analyze this quasi-experiment when randomization is not possible? Walk through how you would estimate causal impact, define success metrics and guardrails, and decide whether the evidence is strong enough to recommend shipping more broadly.
What this tests
- Choosing an identification strategy for non-random treatment assignment
- Defining a primary metric, guardrails, and an explicit MDE
- Reasoning about power limits in observational settings
- Pre-registering analysis to avoid post-hoc fishing
Practicing as: Product Growth Analyst interview at SpliceHi, I'll play your Splice interviewer for the Product Growth Analyst role. Candidates describe these interviews as mostly positive and moderately difficult, so expect me to be friendly and conversational. Take your time with the question above and answer like we're in the room.
You are practicing as a guest. Sign up free to get your answer graded with AI feedback. Your draft stays right here.
