Your question is LLM Ticket Classification and Routing. Start with the requirements and the three tables on the right.
Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.
How do you approach fine-tuning or prompt-engineering an LLM to automatically classify and route incoming user support tickets?
Using the provided ticket, prediction, and queue data, write a query to evaluate which model versions and predicted queues are reliable for automatic routing. Include only predictions with confidence of at least 0.70 and tickets with a known actual queue.
model_version, predicted_queue, eligible_tickets, correct_tickets, accuracy_pct, and avg_confidencemodel_version, accuracy descending, then predicted queue alphabetically| Column | Type | Description |
|---|---|---|
| ticket_idPK | INT | Unique support ticket identifier |
| subject | TEXT | Short ticket subject |
| actual_queue_id | INT | Reviewed queue assignment |
| Column | Type | Description |
|---|---|---|
| prediction_idPK | INT | Unique prediction identifier |
| ticket_id | INT | Ticket being classified |
| model_version | VARCHAR(30) | LLM or prompt version used for prediction |
| predicted_queue_id | INT | Queue selected by the model |
| confidence | DECIMAL(4,3) | Model confidence from 0 to 1 |
| Column | Type | Description |
|---|---|---|
| queue_idPK | INT | Unique support queue identifier |
| queue_name | VARCHAR(80) | Human-readable queue name |