Context
Finwell, a mid-market fintech SaaS company, wants to add an LLM assistant into its customer support workflow. The assistant drafts replies to billing, account access, and policy questions for human agents, and may later be exposed directly to customers for low-risk cases.
Constraints
- p95 latency: 2,500ms for agent-assist responses
- Cost ceiling: $15K/month at 300K requests/month
- Hallucination ceiling: <2% on policy and account-related answers
- Prompt injection success rate: <1% on adversarial tests
- PII and account data must not be exposed outside user authorization scope
- High-risk actions (refunds, account changes, legal commitments) must never be executed autonomously
Available Resources
- 120K support articles, policy docs, macros, and historical resolved tickets
- CRM metadata (customer tier, product, region) available at request time
- Internal search API with BM25 and vector search
- Approved LLMs: GPT-4.1-mini, GPT-4.1, and a cheaper classifier model
- 2,000 labeled support conversations and 300 policy QA pairs
- Human support agents available for escalation and feedback labeling
Task
- Design a risk-aware LLM architecture for this workflow, including where retrieval, classification, and human review should sit before any customer-visible output.
- Define an evaluation plan first: offline and online metrics for hallucination, unsafe advice, prompt injection, escalation quality, and business impact.
- Propose a system prompt and response policy that separates low-risk informational answers from high-risk cases requiring refusal or escalation.
- Estimate cost and latency at target volume, and explain what model-routing or retrieval choices you would make to stay within budget.
- Identify the main failure modes when introducing AI into a customer workflow, and describe concrete mitigations, monitoring, and rollback triggers.
Your answer should focus on practical engineering tradeoffs rather than general AI ethics. Assume the company wants a phased rollout from agent-assist to limited customer-facing automation only if risk metrics are consistently met.
