Business Context

You’re the analytics partner for the Rider App engineering org at Uber. The org ships features for pricing, ETA, and checkout across iOS/Android and a shared backend. The org is ~120 engineers across 10 squads, releasing to production multiple times per day. Reliability is business-critical: checkout regressions can directly impact conversion and driver supply.

Over the last 8 weeks, leadership rolled out two changes: (1) a new code review policy (2 reviewers required for most repos) and (2) a shift from Kanban-like flow to 2-week sprints with a renewed focus on “commitment.”

Metric Scenario

At the QBR, the VP Eng flags a worrying pattern:

• Cycle time (ticket “In Progress” → “Deployed”) increased from 4.2 days to 6.1 days (+45%).
• Velocity (story points completed per squad per sprint) is up from 38 to 44 (+16%).
• Defects are also up: production incidents attributed to Rider App changes rose from 6/week to 10/week, and customer support contacts tagged “app checkout issue” increased 12%.

Product leaders are confused: “If velocity is up, why are we slower and buggier?” Engineering leaders suspect the metrics are being gamed or are measuring different things.

You have one week to deliver a metrics readout and a plan that can be operationalized in the next sprint planning cycle.

Data Available

What You Need To Produce

1. Define cycle time, velocity, and defect rate precisely for this org (including inclusion/exclusion rules and edge cases like blocked work, partial rollouts, and hotfixes).
2. Reconcile the apparent contradiction (velocity up while cycle time and defects worsen). Provide at least 3 plausible mechanisms grounded in how these metrics are constructed.
3. Decompose each metric into actionable drivers (e.g., PR review wait time vs build time vs release queue time; defect mix by severity and component).
4. Propose a diagnostic plan: the first 5 cuts you would run in SQL/BI, and what outcome would confirm/refute each hypothesis.
5. Recommend concrete actions (process or product/engineering changes) and specify guardrail metrics to ensure you don’t “improve” one metric by harming another.

Constraints:

• You cannot change headcount this quarter.
• Any recommendation must keep P0/P1 incident rate from increasing.
• Leadership wants a metric framework that discourages point inflation and encourages true flow efficiency.