Company Context

Databricks is a leading data and AI platform used by enterprises to build data pipelines, run analytics, train models, and govern data assets. Its users range from analysts to highly technical personas such as data engineers, ML engineers, and platform administrators working inside the Databricks Workspace, notebooks, Jobs, and Unity Catalog.

Problem

Databricks wants to improve the experience for a highly specialized user base: data engineers who manage complex production pipelines across notebooks, Jobs, clusters/serverless compute, and data dependencies. Research shows these users are powerful but often forced to stitch together context across multiple surfaces to answer basic operational questions like: "Why did this pipeline fail?", "What changed since the last successful run?", and "Which downstream tables are impacted?"

Internal data shows that for complex failed Jobs, median time-to-diagnosis is 27 minutes, and 38% of users open 4 or more Databricks surfaces before identifying the root cause. Qualitative feedback says the current experience is powerful but fragmented, especially for newer team members onboarding into mature production environments.

Task

1. Define the primary user segment and the core job-to-be-done you would design for.
2. Identify the biggest user pain points in the current Databricks experience and explain how you would validate them.
3. Propose an MVP solution for Databricks that improves the workflow for diagnosing and resolving pipeline issues.
4. Prioritize the most important features and explain the trade-offs you would make given limited design and engineering capacity.
5. Define success metrics and describe how you would know the solution is delivering value to technical users without oversimplifying their workflow.

Constraints

• MVP must ship within one quarter.
• Only one product designer and one small cross-functional engineering pod are available.
• The solution should primarily leverage existing Databricks surfaces rather than introducing an entirely new standalone product.
• The design must serve expert users well while remaining learnable for less experienced team members.