Diagnose Databricks Engagement Drop

Business Context

Databricks has seen a sudden decline in engagement on the Databricks Data Intelligence Platform over the last 14 days. Leadership wants to know whether the drop reflects a real change in user behavior, a measurement issue, or a shift in user mix after recent workspace and onboarding changes.

Metric Scenario

The primary KPI, weekly engaged users, fell from 82,000 to 68,000 (-17.1%) week over week. In the same period, DAU/WAU declined from 0.46 to 0.39, median sessions per active user fell from 5.2 to 4.1, and 7-day retention for new workspace users dropped from 34% to 27%. Top-of-funnel volume was mostly stable: trial signups were down only 2%, and workspace creations were down 1%. A new homepage experience in the Databricks workspace launched 10 days ago, and a tracking library update for product events shipped 12 days ago.

Requirements

1. Define the engagement KPI precisely, including what counts as an engaged user and what events should be included or excluded.
2. Decompose the drop into its main drivers using a clear KPI tree.
3. Distinguish between instrumentation issues, acquisition mix changes, and true product behavior changes.
4. Identify the first 3-5 cuts of data you would review and the hypotheses each cut is testing.
5. Recommend concrete next actions and the guardrail metrics you would monitor.

Data Available

• workspace_events: user_id, workspace_id, event_name, event_time, page_surface, session_id, device_type, browser, country
• workspace_accounts: workspace_id, plan_type, cloud, region, created_at, owner_segment
• user_profiles: user_id, account_created_at, role, persona, company_size, sales_segment
• experiment_exposure: user_id, experiment_name, variant, exposure_time
• marketing_attribution: user_id, acquisition_channel, campaign_id, signup_date
• event_pipeline_monitoring: event_volume_by_day, schema_errors, dropped_events, SDK_version