You are building a text classification system for an enterprise software platform that receives thousands of unstructured support tickets each week from email, chat, and web forms. Each ticket contains noisy free text such as stack traces, product names, abbreviations, pasted logs, and inconsistent formatting, and the operations team wants tickets automatically routed into categories like billing, access, bug report, feature request, and account management. You have roughly 250,000 historical tickets with analyst-assigned labels, but the data is imbalanced, labels have some noise, and new product terminology appears regularly. The system should be practical to retrain and easy to monitor as ticket language changes over time.
How would you design and build this text classification system end to end, including preprocessing, model selection, training, evaluation, and how you would handle noisy labels, class imbalance, and evolving vocabulary in production?