Choose Metrics for Invite Flow

Context

LinkLoop, a professional networking app, wants to redesign its post-signup invite flow to encourage new users to invite colleagues during onboarding. The growth team believes a simpler flow with suggested contacts and fewer steps will increase activation, but they do not want to hurt downstream retention or spam users' address books.

Hypothesis Seed

The proposed treatment replaces the current 3-step invite flow with a 1-step flow that pre-selects recommended contacts and emphasizes the value of building a network early. The team expects this to increase the share of new users who send at least one invite, which may improve activation and week-1 engagement.

Constraints

• Eligible traffic: 120,000 new-user signups per day globally
• Only 60% of signups reach the invite step
• Maximum experiment duration: 14 days, because onboarding engineering resources are locked after that
• False positives are costly: shipping a spammy experience could increase complaints and hurt brand trust
• False negatives are also meaningful: onboarding is a major growth lever, so missing a real improvement delays quarterly targets

Deliverables

1. Define the primary, secondary, and guardrail metrics for this experiment, including the exact metric formula, unit of analysis, baseline, and MDE for the primary metric.
2. State the null and alternative hypotheses, choose the unit of randomization, and justify the allocation and duration.
3. Calculate the required sample size with explicit assumptions (alpha, power, baseline, MDE), and translate it into expected runtime under the available traffic.
4. Pre-register an analysis plan: statistical test, peeking policy, treatment of secondary metrics, and how you will check for sample ratio mismatch.
5. Give a clear ship / don't-ship / iterate recommendation framework that respects guardrails, and name the biggest experimentation risks (for example novelty effects, interference, or SUTVA violations).