Business Context
Blue Cross Blue Shield of Michigan receives a large volume of member messages through the member portal and customer support channels. Your task is to describe and implement an NLP approach that classifies these messages into operational intents so they can be routed faster to the right team.
Data Characteristics
- Volume: ~180,000 historical member messages collected over 18 months
- Text length: 10-350 words, median ~62 words
- Language: Primarily English, with occasional abbreviations, misspellings, and insurance terminology
- Labels: 5 intent classes —
claims_question, benefits_coverage, id_card_request, provider_search, billing_payment
- Distribution: Moderately imbalanced;
claims_question and benefits_coverage make up ~55% of all examples combined
Success Criteria
A good solution should achieve macro-F1 >= 0.82 and recall >= 0.90 for claims_question and billing_payment, since those messages often drive time-sensitive follow-up. The approach should be explainable enough for operations stakeholders and practical to deploy in a secure enterprise environment.
Constraints
- Must run inside Blue Cross Blue Shield of Michigan's secure environment
- Inference should support near-real-time routing for portal submissions
- Avoid heavyweight preprocessing that is hard to maintain
- Solution should handle noisy text, short messages, and domain-specific vocabulary
Requirements
- Build a multi-class text classification pipeline for member message intent.
- Show a realistic preprocessing workflow for insurance-related text.
- Implement a baseline and a transformer-based approach in modern Python.
- Explain how you would fine-tune, validate, and compare the models.
- Describe how you would evaluate class imbalance, confusion patterns, and deployment readiness.
- Note how your prior NLP experience would inform trade-offs between simpler models and fine-tuned language models.
