Business Context
Bhatia Brothers runs several internal digital products, including the Bhatia Brothers Supplier Portal and Bhatia Brothers Service Desk. Over the last two quarters, leadership has seen mixed signals: feature throughput is up, but customer-reported issues and missed delivery commitments have also increased.
Metric Scenario
You are the Engineering Manager for a 28-person team (4 squads) supporting the Supplier Portal. In Q1, the team shipped 96 story points per sprint on average, up from 78 in Q4. However, production incidents rose from 11 to 19, P1/P2 incident resolution time increased from 3.8 hours to 6.1 hours, and the percentage of roadmap commitments delivered on time fell from 84% to 71%. Product leadership wants a clear performance scorecard, while the CTO is concerned that the team may be optimizing for output instead of outcomes, quality, and reliability.
Requirements
- Define the core metrics you would use to evaluate this engineering team, separating leading and lagging indicators.
- Explain which metric or small set of metrics should be the primary KPI(s), and why story points alone are insufficient.
- Decompose the recent performance change to identify whether the main issue is delivery predictability, quality, operational load, or team efficiency.
- Propose a dashboard for weekly and monthly review, including guardrails to prevent gaming.
- Recommend actions you would take if throughput rises while reliability and customer experience worsen.
Data Available
- jira_issues: sprint_id, team_id, issue_type, story_points, status, created_at, resolved_at, planned_flag
- deployments_log: deployment_id, service_name, deployed_at, rollback_flag, change_size
- incident_log: incident_id, severity, service_name, opened_at, resolved_at, root_cause_category
- supplier_portal_usage: account_id, daily_active_users, task_success_rate, page_latency_ms
- service_desk_tickets: ticket_id, product_area, created_at, resolution_time_hours, csat_score
