Business Context

AbbVie is collecting internal survey responses from business teams about their experience with Artificial Intelligence and Large Language Models. The analytics team wants an NLP system that classifies each response into a primary capability area so leaders can quantify adoption patterns and identify where enablement is needed.

Data

You are given 85,000 historical free-text responses from employees across commercial, medical, R&D, and operations teams submitted through an AbbVie internal feedback workflow. Responses are in English, range from 10 to 220 words (median 48), and are labeled into 5 classes: No Practical Experience (18%), Prompting / Summarization (27%), Information Retrieval / Search (21%), Text Analytics / Classification (19%), and Workflow Automation / Copilot Usage (15%). About 6% of responses contain abbreviations, product names, or pharma-specific terminology.

Success Criteria

A good solution should achieve macro-F1 >= 0.82, with recall >= 0.88 for No Practical Experience and Workflow Automation / Copilot Usage so downstream reporting is reliable.

Constraints

• Inference should complete in <150 ms per response in an AbbVie-managed environment.
• The solution must support weekly retraining.
• Avoid sending sensitive internal text to external APIs.
• The model should be explainable enough for business stakeholders to understand common themes.

Requirements

1. Build a multi-class text classification pipeline for employee AI/LLM experience responses.
2. Define a preprocessing workflow for short, noisy business text.
3. Implement a modern Python solution using transformer fine-tuning, with a simple baseline for comparison.
4. Evaluate performance with class-aware metrics and confusion analysis.
5. Describe how you would handle class imbalance, domain vocabulary, and ambiguous responses.