| ENTERPRISE RAG ARCHITECTURE |
+-----------------------------------------------------------------------------------+
[ Unstructured Data ] -> [ Chunking & Tokenization ] -> [ Embedding Model ]
|
v
[ User Prompt ] -> [ Query Transformation ] ---------> [ Vector Database ]
| |
v v
[ Intent Classifier ] -------------> [ Hybrid Search ]
|
v
[ Final Response ] <--- [ LLM Inference Engine ] <--- [ Reranker (Cross-Encoder) ]
**Be ready to go over:**
- **Chunking Strategies** – Tradeoffs between fixed-size, sentence-level, recursive character, and semantic chunking based on document layout.
- **Hybrid Retrieval Systems** – Combining keyword search (BM25) with dense vector search (cosine/dot-product) using Reciprocal Rank Fusion (RRF).
- **Advanced Context Reranking** – Applying cross-encoders to re-rank top-$k$ retrieved chunks before feeding them to the LLM context window.
- **Advanced concepts (less common)** – Contextual compression, GraphRAG for knowledge graph integration, stateful multi-step agent reasoning (e.g., LangGraph), and speculative decoding for low-latency output generation.
**Example questions or scenarios:**
- "A financial services client wants to implement an internal assistant to search through 100,000 dense PDF audit reports. How do you design the ingestion, indexing, and context retrieval pipeline to guarantee less than 2-second response times?"
- "How do you mitigate hallucinations when building a customer support bot that generates answers based strictly on proprietary API documentation?"
### Multi-Agent Systems & LLM Evaluation
Building multi-agent systems and measuring output reliability are critical skills for deploying enterprise AI. Candidates must understand agent orchestration and continuous evaluation frameworks.
**Be ready to go over:**
- **Agent Orchestration** – Structuring autonomous loops, tool calling, memory state management, and fallback mechanisms for complex task decomposition.
- **Evaluation Frameworks** – Implementing LLM-as-a-judge patterns, RAG Triad metrics (Groundness, Answer Relevance, Context Relevance), and automated regression testing.
- **Guardrails & Safety** – Building output filtering, prompt injection defense, and input sanitization layers around deployed model APIs.
- **Advanced concepts (less common)** – Hierarchical multi-agent delegation frameworks, synthetic dataset generation for evaluation benchmarking, and automated fine-tuning feedback loops (RLHF/DPO).
**Example questions or scenarios:**
- "How would you structure a multi-agent system where one agent drafts SQL queries, a second executes them, and a third summarizes the result for business executives?"
- "Describe how you would establish a continuous integration (CI/CD) evaluation test suite for a live enterprise LLM application."
### System Design for LLM Serving & Embeddings
Designing resilient systems requires choosing appropriate data stores, queueing mechanisms, caching layers, and compute strategies for hosting AI services at scale.
**Be ready to go over:**
- **Vector Database Benchmarking** – Comparing index structures (HNSW vs. IVF-PQ) for speed, memory consumption, and recall accuracy.
- **LLM Serving Infrastructure** – Optimizing deployment throughput using dynamic batching, paged attention, KV caching, and model quantization (AWQ, GPTQ).
- **Backend Architecture & APIs** – Designing RESTful and streaming gRPC/WebSocket microservices to stream response tokens smoothly to clients.
- **Advanced concepts (less common)** – Multi-node GPU orchestration, pipeline parallelism versus tensor parallelism, cold start optimization in serverless ML deployments, and distributed vector index sharding.
**Example questions or scenarios:**
- "Design the complete backend architecture for an enterprise AI gateway serving 50 internal applications with variable throughput requirements."
- "Explain your strategy for maintaining high read-throughput on a vector database with 50 million 1536-dimensional vectors subject to frequent batch updates."
