Use Data to Prioritize Onboarding

Business Context

Duolingo launched a redesigned mobile onboarding flow for new learners 6 weeks ago to improve activation and early retention. Since launch, install-to-registration conversion increased from 62% to 68%, but 7-day retention for new users fell from 31% to 26%, and lesson starts per new user in week 1 declined from 5.4 to 4.6.

The Head of Product wants a data-driven recommendation on whether to keep, roll back, or iterate on the new onboarding. Your task is to show how you would use product data to inform that decision rather than relying on anecdotes.

Requirements

1. Define the primary KPI you would use to evaluate onboarding success and explain why it is the right decision metric.
2. Quantify the trade-off between improved top-of-funnel conversion and weaker downstream engagement/retention.
3. Decompose the KPI movement to identify whether the issue is driven by specific platforms, acquisition channels, learner intents, or onboarding steps.
4. State 3-4 hypotheses for why the new flow improved registration but hurt retention, and explain what data would validate each.
5. Recommend a product decision: keep, roll back, or ship a targeted iteration, with clear success criteria and guardrails.

Data Available

• app_installs: install_id, user_id, install_timestamp, platform, country, acquisition_channel, campaign_id
• onboarding_events: user_id, event_timestamp, onboarding_step, step_completed, selected_goal, selected_language, notification_opt_in
• registrations: user_id, registration_timestamp, signup_method
• lesson_events: user_id, lesson_id, lesson_start_timestamp, lesson_complete_timestamp, lesson_type
• user_activity_daily: user_id, activity_date, sessions, lessons_started, lessons_completed, streak_day
• experiments: user_id, variant, exposure_timestamp

Assume the redesign was released to 100% of new users, but you can compare pre/post periods and segment deeply enough to isolate likely drivers.