Leading Indicators Before Retention Impact

Business Context

Duolingo has launched a new AI-powered lesson recap feature to improve long-term learner retention. The feature rolled out to 20% of new users last week, but 4-week retention is the primary success metric and will not be observable for another month.

Metric Scenario

The Head of Growth wants an early read on whether the feature is likely to improve retention without waiting for full retention data. In the first 7 days after launch, exposed users show: lesson completion rate of 61% vs 55% for control, average sessions per user of 4.8 vs 4.2, Day-1 return rate of 41% vs 38%, and recap feature adoption of 46% among exposed users. However, average session length fell from 14.2 minutes to 12.9 minutes, and notification opt-out rose from 9% to 12%.

Stakeholders are asking which leading indicators you would monitor, how you would define them, and how you would decide whether the feature is on track before retention matures.

Requirements

1. Define 4-6 leading indicators you would use as proxies for future retention.
2. Explain why each indicator should predict retention for this product.
3. Identify which of the observed movements are encouraging vs concerning.
4. Propose a simple metric framework with one primary early signal and guardrails.
5. Describe how you would segment the analysis to avoid misleading conclusions.

Data Available

• user_events: lesson_start, lesson_complete, recap_open, recap_complete, app_open, notification_click
• user_daily_activity: user_id, date, sessions, minutes_spent, lessons_completed
• experiment_assignment: user_id, variant, assignment_date, platform, country
• notifications_log: sends, opens, opt_outs
• user_profile: signup_date, acquisition_channel, language_learning_goal, device_type