Databricks recently launched a redesigned Notebook editor and workspace navigation intended to help data practitioners create, run, and share notebooks faster. Four weeks after launch, leadership wants to know whether the design improved both user outcomes and business outcomes, not just whether the new UI was shipped successfully.
Before launch, weekly active notebook creators were 82,000, notebook creation-to-first-run conversion was 68%, median time from notebook open to first successful run was 11.5 minutes, 8-week retention for new workspace users was 41%, and weekly workspace upgrade rate from trial to paid was 3.8%. In the four weeks after launch, weekly active notebook creators rose to 86,500, creation-to-first-run conversion increased to 72%, median time to first successful run fell to 8.9 minutes, but average collaboration actions per creator dropped from 4.6 to 4.1, and trial-to-paid upgrade rate stayed flat at 3.7%. CSAT for the Notebook experience increased from 4.0 to 4.3 / 5.
Stakeholders disagree on whether this redesign was a success: design points to faster task completion and higher satisfaction, while product leadership is concerned that collaboration and monetization did not improve.
notebook_events: notebook_opened, notebook_created, cell_run_started, cell_run_succeeded, notebook_shared, comment_added, dashboard_publishedworkspace_users: user_id, persona, account_age_days, plan_type, workspace_id, countryworkspace_billing: trial_start_date, paid_conversion_date, ARR_band, seat_countux_feedback: CSAT, in-product survey responses, session replay tagsfeature_exposure_log: redesign exposure timestamp, old vs new experience, device_type