Track Engineering Initiative Health

Business Context

Notion is rolling out an engineering initiative to improve workspace performance by reducing page load latency and editor errors. The CTO wants a simple framework to determine whether the initiative is on track before the quarterly review.

Metric Scenario

The initiative has a 12-week target: reduce p95 page load time from 3.8s to 2.5s, reduce editor crash rate from 1.4% to 0.6% of sessions, and improve weekly workspace retention from 71% to 74%. After 5 weeks, p95 load time is 3.1s, crash rate is 1.1%, and retention is still 71.2%. Engineering has shipped 6 of 10 planned milestones, but incident count increased from 4 to 7 per week after the latest release. Product leadership is asking whether the initiative is truly on track or if output is masking weak business impact.

Requirements

1. Define the primary KPI you would use to judge whether the initiative is on track.
2. Identify which metrics are leading indicators versus lagging indicators.
3. Show how you would decompose the KPI to diagnose whether progress is real or concentrated in a narrow segment.
4. Explain what thresholds or trends would cause you to say the initiative is off track.
5. Recommend 3-4 actions or follow-up analyses based on the current numbers.

Data Available

• page_performance_daily: page_id, workspace_id, user_id, device_type, country, load_time_ms, date
• editor_sessions: session_id, workspace_id, user_id, session_start, session_end, crashed_flag, error_count, app_version
• workspace_activity_weekly: workspace_id, week_start, active_users, retained_next_week_flag, plan_tier
• release_log: release_id, release_date, feature_flag, milestone_name, rollout_percent
• incidents: incident_id, start_time, severity, affected_service, root_cause