Scenario

You are adding an LLM assistant to a customer-facing support workflow for a financial services product. The assistant drafts responses to inbound customer questions about account status, required documents, payment timing, and policy explanations, and a human agent can either send the draft or edit it. You handle roughly 8,000 conversations per day, and leadership wants faster first-response times without increasing compliance risk. Because customers may act on the assistant's answers, incorrect or overconfident responses are considered high severity.

Constraints

• p95 latency: 2,500ms per draft response
• Cost ceiling: $0.03 per conversation turn
• Hallucinated policy or account-specific claims must stay below 1% on a reviewed golden set
• The system must not reveal PII, internal instructions, or unsupported financial guidance
• Suspicious or low-confidence cases must escalate to a human instead of guessing

Available Resources

• Historical support conversations, policy documents, and approved response templates
• Access to a hosted LLM API and embeddings API
• A small labeling budget for expert review of 1,000 examples per month
• Existing customer metadata and permissions checks from the support platform

Question

How would you design this customer-facing LLM workflow so that it is useful but safe, and how would you evaluate and mitigate the major risks before launch and after rollout?