Business Context

Google has launched a new Gmail inbox UI on web, changing navigation placement, message list density, and the compose entry point. Product leadership wants to know whether the redesign improves or harms long-term user retention, not just short-term clicks.

Metric Scenario

The UI was rolled out to 20% of eligible signed-in Gmail web users in the US for 6 weeks. Early readouts show mixed signals: average sessions per user increased from 5.8 to 6.1 per week in the exposed group, but average session duration fell from 7.4 to 6.6 minutes. Week-1 return rate was 68% for exposed users vs 67% for control, but stakeholders are concerned this may not translate into durable retention. Gmail’s PM asks you to define the right retention metric, identify leading indicators that can predict long-term impact sooner, and explain how you would interpret conflicting engagement movements.

Requirements

1. Define the primary long-term retention metric for this UI change, including the user population, time window, and numerator/denominator.
2. Propose 2-3 leading indicators that should be monitored before full long-term retention data matures.
3. Describe how you would compare exposed vs control users and what cuts of the data you would review first.
4. Explain how you would distinguish a true retention improvement from a novelty effect or a shift in usage intensity.
5. Recommend launch criteria, including at least one guardrail metric that should prevent rollout.

Data Available

• gmail_web_exposure_log: user_id, exposure_date, experiment_group, browser, country, account_age_days
• gmail_web_sessions: user_id, session_start, session_end, page_views, compose_clicks, message_opens, searches
• gmail_core_actions_daily: user_id, date, emails_sent, emails_replied, emails_archived, labels_created
• user_activity_summary: user_id, week_start, active_days, sessions, total_minutes
• support_feedback_log: user_id, date, help_center_visits, complaint_tag, ui_feedback_score