Business Context

Motive recently redesigned the onboarding flow in the Motive Driver App for newly invited drivers. The new flow launched on April 1 and added a guided setup checklist for HOS, vehicle inspection, and document upload. The PM wants to know whether this product change improved retention for newly activated drivers.

Metric Scenario

Before launch, the 4-week retention rate for new driver accounts activated in February was 31% and in March was 32%. For the April activation cohort, preliminary 4-week retention is 37%. However, April also had a higher share of enterprise fleets, more Android users, and a 15% increase in activation volume due to a sales push. Leadership wants to know whether the retention improvement is real, which cohorts drove it, and whether the onboarding change improved downstream engagement in the Motive Driver App.

Requirements

1. Define the retention metric precisely, including the cohort start event, the return event, and the retention window.
2. Explain how you would structure cohort analysis to compare pre-change and post-change cohorts fairly.
3. Identify which cohort cuts you would analyze to determine whether the lift is broad-based or concentrated in specific segments.
4. Describe how you would separate the effect of the onboarding change from mix shift, seasonality, or acquisition-quality changes.
5. Recommend what conclusions you would present to the PM if retention improved overall but worsened for some important subgroups.

Data Available

• driver_accounts: driver_id, fleet_id, activation_date, country, device_os, app_version
• driver_app_events: driver_id, event_time, event_name, session_id, screen_name
• fleet_accounts: fleet_id, segment, fleet_size, sales_channel, contract_start_date
• onboarding_checklist_events: driver_id, checklist_step, completed_at
• hos_logs and inspection_events: driver_id, event_time, status
• experiments_and_releases: release_date, app_version, feature_flag exposure