Choose Metrics for Ticket Escalation

Context

Abzooba has built a binary classifier in its customer support workflow to predict whether an incoming enterprise support ticket should be escalated to a senior resolution team. The current model is used in production to prioritize tickets in Abzooba’s support intelligence stack, but stakeholders disagree on whether accuracy, precision, recall, or F1 should be the primary success metric.

Current Performance

The model was evaluated on 10,000 labeled tickets, of which 1,200 truly required escalation.

Confusion matrix counts:

The Problem

The Head of Support points to 90% accuracy and argues the model is strong. The senior operations manager is more concerned that 516 high-priority tickets are missed, causing SLA breaches and delayed enterprise responses. Meanwhile, the support team also wants to avoid too many unnecessary escalations because senior agents are capacity constrained.

Your Task

1. Explain how you would choose between accuracy, precision, recall, and F1 for this use case.
2. Interpret the current metrics in business terms, not just mathematically.
3. Identify which metric should be optimized first under the stated constraints.
4. Recommend what additional analysis you would run before changing the model or threshold.
5. Propose concrete next steps to improve performance.

Constraints

1. Senior escalation team can handle at most 1,300 tickets/day.
2. Missing a true escalation is estimated to cost 8x more than an unnecessary escalation.
3. Product leadership wants a single primary metric for model reporting.