Hebbia Chat System Design

Design 'Hebbia Chat': a chat interface where users retrieve insights from SEC filings, built for speed and durability at scale, using a multi-agent system with an orchestrator agent that decides which sub-agent(s) to call. Loading states must be shown before/after every agent call, output must be streamed to the frontend, users can cancel a message mid-flight, and there is a finite token budget to plan around.

Asked in the technical phone screen stage. This is a practical multi-agent system-design question modeled on Hebbia's SEC-filings retrieval product. Clarify missing scale, latency, availability, and token-budget targets with the interviewer.

Address:

1. Request lifecycle, orchestration, and agent contracts.
2. SEC-filings ingestion, retrieval, ranking, citation grounding, and durable storage.
3. Streaming, loading states, cancellation, retries, timeouts, and partial results.
4. Token-budget allocation and safeguards against runaway agent execution.
5. Offline and online evaluation, monitoring, drift, skew, and failure modes.