Growth Invite Flow Experiment

Context

LoopChat, a consumer messaging app, wants to launch a new growth feature: after a user sends their first message of the day, they see a lightweight prompt to invite friends via contacts. The growth team believes this will increase new-user acquisition, but product leadership is concerned it could hurt engagement if the prompt feels spammy.

Hypothesis Seed

The treatment adds an in-app invite prompt with one-tap sharing and personalized copy. The team expects it to increase successful invites and downstream activated signups, while keeping core engagement and retention stable.

Constraints

• Eligible traffic: 120,000 existing active users per day
• Average eligibility rate: 1 exposure opportunity per eligible user per day
• Baseline successful invite rate: 8.0% of eligible users send at least one invite within 24 hours
• Maximum experiment duration: 21 days
• Decision deadline is fixed because marketing needs a launch recommendation before the next campaign
• False positives are costly because an annoying growth surface could reduce long-term retention; false negatives are acceptable if modest

Tasks

1. Define the primary metric, 2-4 guardrail metrics, and a realistic MDE for this growth feature.
2. Calculate the required sample size and determine whether the test can be completed within 21 days given the available traffic.
3. Choose the unit of randomization, allocation, duration, and any stratification/blocking you would use.
4. Pre-register an analysis plan: statistical test, peeking policy, multiple-comparison policy, and how you will handle any mismatch between unit of randomization and analysis.
5. State a clear ship / don’t-ship / iterate rule that respects guardrails, and identify the main pitfalls for this experiment.