Context
BeaconCMS wants an event-driven AI platform for asynchronous tasks such as product-description generation, metadata enrichment, moderation summaries, and taxonomy tagging. Requests come from multiple internal products and must be processed reliably without blocking user-facing APIs.
Constraints
- API acknowledgment to producer: <150ms p95
- End-to-end completion SLA: 95% of jobs within 2 minutes; 99% within 10 minutes
- Cost ceiling: $35K/month at 8M jobs/month
- Hallucination ceiling: <2% for grounded enrichment fields on a 1,000-job golden set
- Prompt injection success rate: <0.5% on adversarial inputs
- PII must not be persisted in prompts, logs, or dead-letter payloads
- System must support retries, idempotency, partial failures, and human review for high-risk outputs
Available Resources
- Event bus (Kafka or SNS/SQS), object store, Postgres, Redis, and a workflow engine
- Input payloads include raw content, product metadata, locale, customer tier, and optional reference documents
- Approved LLMs: a low-cost small model for classification/extraction and a higher-quality model for generation
- Existing policy service for content safety and a rules engine for deterministic validations
- 50K historical jobs with human-reviewed outcomes; 1,000 examples can be labeled for a golden set
Task
- Design the event-driven architecture for job intake, orchestration, retries, idempotency, state tracking, and result delivery across multiple asynchronous AI tasks.
- Specify how prompts, tool calls, and structured outputs differ for generation vs enrichment jobs, including refusal behavior and safety checks.
- Define an evaluation-first plan: offline quality/safety evaluation before launch, then online monitoring, alerting, and rollback criteria.
- Estimate cost and latency by stage, and explain routing decisions between small vs large models under the monthly budget.
- Identify key failure modes such as hallucinated fields, prompt injection in source content, duplicate processing, and stuck jobs, with concrete mitigations.
