Business Context
NovaDocs, a developer-tools company, wants to automatically review AI-generated explanations of technical concepts before publishing them in its beginner help center. Your task is to build an NLP system that classifies whether an explanation of "RESTful API" is understandable for someone with no technology background.
Data
- Volume: 180,000 labeled explanations collected from internal writers, LLM outputs, and historical support content
- Text length: 40-350 words, typically 1-3 short paragraphs
- Language: English only
- Labels:
beginner_friendly (62%), too_technical (38%)
- Annotation notes: Explanations are labeled based on jargon use, clarity, analogy quality, and whether they assume prior technical knowledge
Examples range from plain-language analogies ("like ordering food from a menu") to explanations filled with terms such as endpoints, statelessness, JSON, and HTTP verbs.
Success Criteria
A good solution should achieve F1 >= 0.88 on the too_technical class and precision >= 0.90 for beginner_friendly, so low-quality explanations are not mistakenly approved.
Constraints
- Inference latency must stay below 80ms per explanation in batch review workflows
- The model must run on a single CPU service or modest GPU
- The team needs interpretable outputs to explain why a response was flagged
Requirements
- Build a binary NLP classifier for explanation quality:
beginner_friendly vs too_technical
- Design a preprocessing pipeline that handles punctuation, formatting, and domain jargon consistently
- Implement a modern Python solution, including training, validation, and evaluation
- Compare a lightweight baseline with a transformer-based approach
- Describe how you would identify explanations that sound fluent but still assume technical knowledge
