Context
FinBank wants an internal assistant that helps employees summarize security policies, answer vendor-risk questions, and draft incident-response communications using the OpenAI API. The CISO has asked for a practical security assessment before approving production use.
Constraints
- Audience: CISO and security leadership, not ML researchers
- p95 latency: 2,500ms for interactive requests
- Cost ceiling: $12K/month at 400K requests/month
- Hallucination ceiling: <2% on a 200-question security-policy golden set
- No secrets, customer PII, or regulated data may be exposed to unauthorized users or retained outside policy
- System must resist prompt injection in user input and retrieved documents
- Responses used for security decisions must be grounded, auditable, and able to refuse when evidence is insufficient
Available Resources
- 25K internal security documents: policies, standards, control mappings, incident runbooks, vendor assessments
- OpenAI API access with approved enterprise controls
- Existing IAM, DLP, SIEM, and document access-control metadata
- 2 security engineers and 1 compliance analyst available to label eval data
- Option to use a small non-LLM classifier for PII/secrets detection before model calls
Task
- Explain to the CISO the main security implications of using the OpenAI API, including data handling, retention, access control, prompt injection, hallucination, and auditability.
- Design a secure prompting and response policy for this assistant, including refusal behavior and structured outputs for risk-sensitive answers.
- Define an eval-first plan: offline tests for hallucination, prompt injection, and data leakage, plus online monitoring and guardrails after launch.
- Propose a production architecture that minimizes security risk while meeting latency and cost constraints.
- Estimate cost/latency and identify the top failure modes, detections, and mitigations you would present in a CISO review.
