At NovaDesk, a customer support routing model predicts whether an incoming ticket should be marked urgent and escalated to the priority queue. The team recently noticed more customer complaints about delayed handling, even though the model’s overall accuracy still looks acceptable.
You are given the latest evaluation results for the production logistic regression classifier on a 20,000-ticket labeled validation set.
| Metric | Previous Model | Current Model | Change |
|---|---|---|---|
| Accuracy | 0.89 | 0.86 | -0.03 |
| Precision | 0.72 | 0.81 | +0.09 |
| Recall | 0.68 | 0.41 | -0.27 |
| F1 Score | 0.70 | 0.55 | -0.15 |
| AUC-ROC | 0.84 | 0.82 | -0.02 |
| Urgent ticket rate | 18% | 18% | 0 |
| Tickets flagged urgent | 3,060 | 1,820 | -1,240 |
The model is now much more conservative: precision improved, but recall dropped sharply. As a result, many truly urgent tickets are not being escalated, increasing SLA breaches and customer dissatisfaction.