Business Context

Nimbus is a B2B workflow SaaS company with 320 employees and 55 engineers across 6 product squads. The CTO says engineering output looks high, but customer-facing reliability and delivery confidence have become inconsistent over the last two quarters.

Metric Scenario

In Q2, the team shipped 148 production deployments, up from 110 in Q1 (+35%). However, Sev-1 incidents increased from 3 to 8, average lead time for changes rose from 2.4 days to 4.1 days, and sprint commitment reliability fell from 84% to 68%. Product leadership is asking for a clear framework to measure engineering team success without over-indexing on velocity alone. Customer outcomes also moved: enterprise logo churn increased from 1.8% to 2.6%, and NPS for product reliability dropped from 41 to 34.

You are asked to define the right KPI set for the engineering organization and explain how to interpret trade-offs between speed, quality, reliability, and business impact.

Requirements

1. Define a primary success metric for the engineering team and explain why it should be the top-line KPI.
2. Propose 4-6 supporting metrics, separating leading vs lagging indicators.
3. Show how you would diagnose whether Q2 performance actually improved or deteriorated overall.
4. Explain which metrics should be tracked at company, org, and squad level.
5. Identify risks of using a single metric such as story points shipped or deployment count.

Data Available

• deployments: deployment_id, service_id, squad_id, deployed_at, rollback_flag, change_size
• incidents: incident_id, severity, service_id, started_at, resolved_at, root_cause
• jira_issues: issue_id, squad_id, issue_type, story_points, created_at, completed_at, sprint_id
• sprints: sprint_id, squad_id, committed_points, completed_points, sprint_start, sprint_end
• customer_accounts: account_id, segment, churned_flag, renewal_date, nps_reliability_score
• engineering_survey: engineer_id, squad_id, quarter, engagement_score, burnout_risk