Analyze Sentiment in Claims Feedback

Business Context

American Family Insurance - Colorado collects customer feedback from post-claim surveys, call-center notes, email responses, and mobile app comments. The marketing analytics team wants an NLP pipeline that scores sentiment reliably so they can identify friction in the claims experience and track brand perception by touchpoint.

Data

You are given a historical dataset of approximately 420,000 feedback records from the last 18 months across AmFam Colorado channels. Text is mostly English, with about 7% Spanish and occasional insurance-specific abbreviations (e.g., "FNOL," "adjuster," "deductible," "total loss"). Feedback length ranges from 5 to 900 characters, with a median of 110 characters. Labels are available for 120,000 records and follow a 3-class distribution: positive (52%), neutral (28%), negative (20%). Some records contain personally identifiable information and policy references.

Success Criteria

A strong solution should achieve macro-F1 ≥ 0.82, negative-class recall ≥ 0.88, and produce batch scores fast enough to process daily feedback within 30 minutes. Outputs should support dashboarding by channel, product line, and claim stage.

Constraints

• Data must remain in AmFam Colorado's secure environment
• Inference should be efficient enough for daily batch scoring on modest GPU or CPU resources
• The approach should handle class imbalance, short texts, and domain-specific vocabulary
• The solution should be explainable enough for marketing and customer-experience stakeholders

Requirements

1. Build a 3-class sentiment classifier for customer feedback.
2. Define a preprocessing pipeline for noisy insurance text, multilingual records, and PII removal.
3. Implement a modern Python solution, including tokenization, training, and evaluation.
4. Compare at least one lightweight baseline against a transformer-based model.
5. Describe how you would monitor drift and sentiment shifts after deployment.
6. Explain how predictions would be aggregated into reporting for AmFam Colorado stakeholders.