Context
LexiDraft is launching an AI assistant that generates first-pass legal documents for small-business customers, including NDAs, contractor agreements, and basic employment letters. The feature must explain the value of AI-generated legal documents to non-technical buyers without overstating accuracy or implying the output is a substitute for licensed legal advice.
Constraints
- p95 latency: 1,500ms for the explanation response
- Cost ceiling: $6K/month at 120K explanation requests
- Hallucination ceiling: <2% on a 300-prompt golden set, measured as unsupported claims about legal validity, jurisdiction coverage, or review guarantees
- Safety requirement: must not claim the document is legally sufficient in all cases, must not provide unauthorized legal advice, and must clearly disclose limitations
- Tone requirement: plain English, 8th-grade reading level, professional but reassuring
Available Resources
- Product factsheet with approved claims about speed, consistency, editability, and lawyer-review workflows
- A policy document listing prohibited claims and required disclaimers
- 2,000 historical sales-chat transcripts labeled for customer objections and successful explanations
- Access to an approved LLM API and JSON-schema structured output
- Legal/compliance team available to label 300 evaluation prompts
Task
- Design a prompt-based solution that generates customer-friendly explanations of the product's value while staying within legal/compliance boundaries.
- Define an evaluation plan first: offline golden-set scoring, hallucination checks, readability checks, and online metrics after launch.
- Specify the prompt, output schema, and refusal/deferral behavior for questions that cross into legal advice.
- Estimate cost and latency at the target volume, and explain any model-routing or caching strategy.
- Identify likely failure modes, including overclaiming, vague disclaimers, and prompt injection from user-provided text pasted into the chat.
