Problem
Scenario
You want to estimate the impact of a feature launch when you could not randomize users. The feature was rolled out first to a treated group and not to a comparable untreated group. In the 4 weeks before launch, the treated group had 18,400 active users and averaged 2.10 weekly creations per user, while the untreated group had 21,600 active users and averaged 1.95. In the 4 weeks after launch, the treated group averaged 2.42 and the untreated group averaged 2.08. The standard error of the estimated difference-in-differences effect from a regression is 0.041, and you are testing at the 5% level.
Question
How would you estimate the causal effect with a quasi-experimental design here, test whether the effect is statistically significant, and explain what assumptions you would need for the estimate to be credible?
Practicing as: Product Growth Analyst interview at SpliceHi, I'll play your Splice interviewer for the Product Growth Analyst role. Candidates describe these interviews as mostly positive and moderately difficult, so expect me to be friendly and conversational. Take your time with the question above and answer like we're in the room.
You are practicing as a guest. Sign up free to get your answer graded with AI feedback. Your draft stays right here.
