Project Context
Meta's Instagram Reels team has a promising experiment: adding a more prominent Instagram Save entry point on Reels increased saves and downstream revisit behavior in a small internal test. You are the Product Growth Analyst partnering with a PM, EM, 6 engineers, 1 data scientist, and 1 UXR to decide whether and how to scale the test to a broader launch within 8 weeks before Q3 planning closes.
The business goal is to improve the AARRR funnel for Reels by increasing activation and retention without hurting session quality or creator distribution. Leadership also wants to understand whether higher saves translate into better long-term engagement, not just a short-term novelty effect.
Key Stakeholders
- Reels PM wants a fast launch to hit quarterly growth goals.
- Integrity lead is concerned the change may amplify low-quality save-bait content.
- Infra engineering manager has limited logging capacity and wants minimal instrumentation changes.
- FB Groups growth lead wants the same analyst support for a parallel sharing initiative, creating a resourcing conflict.
- Director of Growth expects a clear recommendation backed by experiment rigor.
Constraints
- Timeline: 8 weeks to recommendation and, if approved, phased rollout plan
- Experiment budget: enough for only one additional 10% holdout and 2 engineer-weeks of logging work
- Current test covers 2 markets and 4.8M eligible users
- Existing readouts show a +6.1% lift in save metric, but retention impact is inconclusive
- There is a suspected Sample Ratio Mismatch (SRM) in Android traffic during week 1
Complications
- The PM wants to launch globally based on the save lift, while the data scientist argues the result may be inflated by CUPED setup issues and novelty.
- Reels has network effects, so the team asks whether changes in sharing and the k-factor could bias interpretation beyond direct save behavior.
Deliverables
- Build an 8-week execution plan for decision-making, validation, and launch readiness.
- Define the success criteria, including primary metrics, guardrails, and how to assess novelty vs durable impact.
- Recommend how to handle the SRM issue and whether to rerun, extend, or phase the experiment.
- Present a stakeholder plan that resolves the PM, integrity, and infra trade-offs.
- Propose a phased rollout and rollback plan if leadership approves launch.
