Dataford
Interview QuestionsInterview GuidesExperiencesMock InterviewsPricing
Get started

Causal Lift in Growth Experiment

HardA/B Testing & Experimentation00:00
Practice interviewer
In session
5 left
00:00

Your question is Causal Lift in Growth Experiment. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

You need to log in / sign up to chat or submit.

Problem

Scenario

You work on a growth surface in a B2B SaaS product and launch a new onboarding prompt intended to improve activation. Early results look better in treatment, but you need to know whether the initiative actually caused the improvement rather than reflecting noise, mix shifts, or measurement issues.

Question

How would you determine whether the growth initiative truly caused an improvement? Walk through how you would design the experiment, define success, and decide whether the observed lift is real enough to ship.

Metrics to Define

  • Primary: 7-day activation rate
  • Guardrails: retained active accounts, frustration/dismissal rate, support contact rate
  • Secondary: time to activation and segment cuts

Design Considerations

  • Randomize at eligible_account_id
  • Use 50/50 allocation
  • Run long enough for a full 7-day outcome window
  • Stratify by acquisition channel and company size