Context
FinSure, a global insurance company, wants to deploy OpenAI API-powered assistants for internal policy search, claims summarization, and customer-support drafting. You are advising the customer on enterprise security, privacy, and compliance requirements before production rollout.
Constraints
- p95 latency: 2,500ms for interactive use cases
- Cost ceiling: $60K/month across 3M requests
- Hallucination ceiling: <2% materially incorrect responses on a 400-task golden set
- No raw PCI, PHI, or government ID numbers may be persisted in prompts, logs, or analytics systems
- Must support regional data controls, auditability, RBAC, and incident response
- System must resist prompt injection from retrieved documents and malicious end users
Available Resources
- OpenAI API access with enterprise controls and approved regional deployment options
- 200K internal documents across policies, SOPs, legal guidance, and compliance manuals
- Existing IAM, DLP, SIEM, KMS, and document-permission systems
- Security team can label 100 adversarial prompts; compliance team can review a 400-task golden set
- You may use prompt design, retrieval, structured outputs, and policy enforcement services, but should avoid unnecessary fine-tuning unless justified
Task
- Design a secure enterprise architecture for OpenAI API usage, including data flow, privacy controls, logging, key management, access control, and document-permission enforcement.
- Write a system prompt for a compliance-aware assistant that answers only from approved sources, refuses unsafe requests, and emits structured metadata for audit review.
- Define an evaluation plan first: offline and online metrics for factuality, refusal quality, prompt-injection resistance, PII leakage, and policy compliance.
- Recommend deployment guardrails for different risk tiers (internal drafting vs. customer-facing responses), including human review, model selection, and fallback behavior.
- Estimate cost and latency, then explain the main tradeoffs between stronger controls, answer quality, and operational overhead.
