Dataford
Interview QuestionsInterview GuidesExperiencesMock InterviewsPricing
Get started

Measure Retention After Gmail Redesign

EasyMetrics00:00
Practice interviewer
In session
5 left
00:00

Your question is Measure Retention After Gmail Redesign. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

You need to log in / sign up to chat or submit.

Problem

Business Context

Google has launched a new Gmail inbox UI on web, changing navigation placement, message list density, and the compose entry point. Product leadership wants to know whether the redesign improves or harms long-term user retention, not just short-term clicks.

Metric Scenario

The UI was rolled out to 20% of eligible signed-in Gmail web users in the US for 6 weeks. Early readouts show mixed signals: average sessions per user increased from 5.8 to 6.1 per week in the exposed group, but average session duration fell from 7.4 to 6.6 minutes. Week-1 return rate was 68% for exposed users vs 67% for control, but stakeholders are concerned this may not translate into durable retention. Gmail’s PM asks you to define the right retention metric, identify leading indicators that can predict long-term impact sooner, and explain how you would interpret conflicting engagement movements.

Requirements

  1. Define the primary long-term retention metric for this UI change, including the user population, time window, and numerator/denominator.
  2. Propose 2-3 leading indicators that should be monitored before full long-term retention data matures.
  3. Describe how you would compare exposed vs control users and what cuts of the data you would review first.
  4. Explain how you would distinguish a true retention improvement from a novelty effect or a shift in usage intensity.
  5. Recommend launch criteria, including at least one guardrail metric that should prevent rollout.

Data Available

  • gmail_web_exposure_log: user_id, exposure_date, experiment_group, browser, country, account_age_days
  • gmail_web_sessions: user_id, session_start, session_end, page_views, compose_clicks, message_opens, searches
  • gmail_core_actions_daily: user_id, date, emails_sent, emails_replied, emails_archived, labels_created
  • user_activity_summary: user_id, week_start, active_days, sessions, total_minutes
  • support_feedback_log: user_id, date, help_center_visits, complaint_tag, ui_feedback_score