- Reranking mechanisms using cross-encoder models to improve retrieval precision.
- Advanced concepts (less common) – GraphRAG integrations, hierarchical indexing, and self-querying retrievers.
Example questions or scenarios:
- "How would you design a RAG pipeline that handles constantly updating legal contracts without re-embedding the entire database?"
- "What steps do you take when your retrieval system returns irrelevant context chunks to the LLM?"
LLM Evaluation
Evaluating non-deterministic generative models requires a sophisticated mix of automated metrics, reference-based testing, and LLM-as-a-judge frameworks. Interviewers want to see that you do not rely solely on manual inspection, but instead build rigorous evaluation pipelines for hallucination, toxicity, and task accuracy. A strong candidate establishes clear quantitative benchmarks tied directly to business requirements.
Be ready to go over:
- Automated evaluation frameworks like Ragas, TruLens, and DeepEval.
- Measuring faithfulness, answer relevance, and context recall in RAG applications.
- Designing human-in-the-loop evaluation workflows for enterprise deployments.
- Advanced concepts (less common) – Adversarial prompt testing, red-teaming methodologies, and custom reward model training.
Example questions or scenarios:
- "Walk me through how you would set up a continuous evaluation pipeline for a customer support chatbot in production."
- "How do you detect and quantify hallucinations when there is no ground-truth reference dataset available?"
Multi-Agent Systems
Enterprise workflows often require more than a single LLM prompt; they demand orchestrated networks of specialized agents capable of planning, executing, and validating tasks. Interviewers will test your understanding of agentic design patterns, state management, and error recovery in autonomous loops. You should be prepared to discuss how agents communicate, share memory, and avoid infinite execution cycles.
Be ready to go over:
- Agent orchestration frameworks like LangGraph, AutoGen, and CrewAI.
- State management, persistent memory stores, and tool-use integration.
- Guardrails, human approval gates, and loop prevention strategies.
- Advanced concepts (less common) – Decentralized agent negotiation, hierarchical planning trees, and dynamic agent spawning.
Example questions or scenarios:
- "Design a multi-agent system where one agent writes code, a second tests it, and a third audits security compliance."
- "How do you handle error propagation when an intermediate agent in a multi-step workflow returns a malformed response?"
Embeddings and Vector Search
Vector embeddings form the bedrock of semantic search and retrieval systems. Interviewers expect you to understand vector space mathematics, distance metrics, and the operational characteristics of modern vector databases. Strong candidates can explain indexing tradeoffs and how to scale vector search to millions of high-dimensional records.
Be ready to go over:
- Choice of distance metrics (Cosine, Dot Product, L2 Euclidean) and their impact on retrieval.
- Approximate Nearest Neighbor (ANN) algorithms such as HNSW and IVF.
- Vector database scaling, sharding, and persistence considerations.
- Advanced concepts (less common) – Product quantization for compression, sparse-dense hybrid embedding alignment, and dimensionality reduction techniques.
Example questions or scenarios:
- "When would you choose HNSW over IVF indexing in a production vector database, and what are the memory trade-offs?"
- "How do you manage embedding model upgrades and version transitions across a massive existing vector index?"
System Design for LLM Serving
Serving large language models at scale presents unique infrastructure challenges, including high memory bandwidth requirements, dynamic batching, and strict latency SLAs. Interviewers will evaluate your ability to design robust serving architectures that balance throughput, cost, and response time. You should be comfortable discussing inference optimization techniques and cloud deployment topologies.
Be ready to go over:
- Serving frameworks and engines like vLLM, TensorRT-LLM, and Triton Inference Server.
- PagedAttention and dynamic batching to maximize GPU utilization.
- Quantization methods (AWQ, GPTQ, GGUF) and their impact on model accuracy and speed.
- Advanced concepts (less common) – Speculative decoding, multi-node tensor parallelisms, and serverless LLM scaling architectures.
Example questions or scenarios:
- "How would you architect an inference service to minimize time-to-first-token for a high-concurrency enterprise application?"
- "Walk through your strategy for monitoring GPU memory utilization and request throttling under unexpected traffic surges."
