Diagnose Weekend Classification Drift

Context

NewsPulse uses a text classification model to route incoming customer support messages into one of two classes: urgent escalation or standard queue. The model performs well Monday through Friday, but support leaders report a sharp quality drop on weekends, causing delayed handling of urgent tickets.

Current Performance

The Problem

Weekend recall for urgent tickets has fallen to 57%, meaning the model misses a large share of truly urgent messages when staffing is already thinner. Product and operations teams want to know whether this is caused by data drift, label quality, traffic mix changes, threshold issues, or a pipeline problem.

Requirements

1. Diagnose the most likely reasons weekend performance is worse than weekday performance.
2. Explain what the metric pattern suggests about model behavior, especially the drop in recall.
3. Propose a structured error analysis plan, including what slices and examples you would inspect first.
4. Recommend specific fixes across data, features, thresholding, retraining, and monitoring.
5. Describe how you would validate that the fix improves weekend performance without hurting weekday quality.

Constraints

• False negatives are more costly than false positives because missed urgent tickets breach SLA.
• Weekend labeled data arrives with a 48-hour delay.
• The model must keep weekday precision above 0.80 after any change.