| 1. System Design & ML Architecture| 2. Multi-Agent Systems & Tools |
| - Hybrid RAG Pipelines | - State Machines & Memory |
| - LLM Fleet Gateways | - Tool Calling & Schemas |
+---------------------------------+---------------------------------+
| 3. Embedding & Vector Search | 4. LLM Evaluation & Telemetry |
| - Indexing (HNSW, IVF-PQ) | - Quantitative Evals (Ragas) |
| - Hybrid Dense/Sparse Search | - Production Traces & Logs |
+---------------------------------+---------------------------------+
| 5. Hands-on Coding & SQL | |
| - Data Pipelines & Crawlers | |
| - Analytical SQL Queries | |
+---------------------------------+---------------------------------+
#### 1. RAG Pipeline & Distributed System Design
Enterprise RAG architectures at **Salesforce** require moving beyond naive document chunking into high-throughput, fault-tolerant vector retrieval pipelines. You will be evaluated on your ability to design systems that handle massive data ingestion, process dynamic permission control, maintain index freshness, and achieve strict sub-second end-to-end response times.
**Be ready to go over:**
- **Hybrid Search Architecture** – Combining dense vector search (semantic similarity) with sparse keyword search (BM25) using Reciprocal Rank Fusion (RRF) to optimize recall across specialized domain terms.
- **Chunking Strategies & Context Re-ranking** – Document parsing, overlapping strategy, semantic chunking, and utilizing cross-encoder re-rankers (e.g., Cohere ReRank) to refine retrieved contexts before prompt assembly.
- **AWS Infrastructure Integration** – Architecting serverless ingestion and retrieval pipelines using AWS Lambda, ECS task workers, S3 document storage, and Managed Vector Databases.
**Example questions or scenarios:**
- "Design an enterprise RAG system for **Salesforce** Knowledge Articles that supports real-time record updates, enforces record-level user access permissions, and achieves sub-200ms retrieval latency."
- "How do you mitigate context compression loss and the 'lost-in-the-middle' phenomenon when feeding massive retrieved context chunks into an LLM prompt?"
- "Architect a high-concurrency Collaborative Spreadsheet System that processes user formulas concurrently while feeding structural updates asynchronously into an AI contextual analysis worker."
#### 2. Multi-Agent Systems & Tool Orchestration
Building autonomous capabilities into **Agentforce** requires sophisticated control flow, durable execution state, precise tool definition, and safe failure recovery mechanisms. Candidates must demonstrate deep knowledge of agent execution loops rather than basic static prompting.
**Be ready to go over:**
- **State Machines & Memory Management** – Designing persistent memory patterns (short-term execution scratchpads, working memory buffers, and long-term vector-backed memory) across multi-turn tool conversations.
- **Structured Output & Schema Enforcement** – Guaranteeing rigid JSON/Pydantic output formatting from non-deterministic LLMs for dynamic tool calling and API invocation.
- **Agent Coordination Patterns** – Orchestrating hierarchical manager-worker patterns, sequential pipeline routing, and consensus-driven multi-agent networks.
**Advanced concepts (less common):**
- Human-in-the-loop (HITL) approval queue integration for high-risk transactional tool execution.
- Infinite execution loop detection, step budgets, and deterministic fallbacks in multi-agent routing.
**Example questions or scenarios:**
- "Walk through the state persistence, retry logic, and tool declaration design for an **Agentforce** deployment that automatically creates and updates sales opportunity records via enterprise APIs."
- "How would you handle non-deterministic outputs or schema validation failures when an AI agent attempts to invoke a critical database mutation tool?"
#### 3. Embeddings & Vector Search Engineering
Retrieval quality depends heavily on deep theoretical and practical understanding of vector representations, indexing choices, and semantic representation limits. You must be comfortable discussing mathematical similarity measures and physical index design.
**Be ready to go over:**
- **Similarity Metrics** – Cosine similarity, Dot Product, and Euclidean distance ($L_2$) trade-offs, alongside vector normalization strategies.
- **Approximate Nearest Neighbor (ANN) Indexing** – Internal mechanics, memory trade-offs, build time vs. search throughput parameters for HNSW (Hierarchical Navigable Small World) graphs and IVF-PQ (Inverted File with Product Quantization).
- **Domain-Specific Fine-tuning** – Contrastive learning techniques (e.g., InfoNCE loss) and fine-tuning embedding models on specialized enterprise dataset domain pairs.
**Example questions or scenarios:**
- "Compare HNSW and IVF-PQ indexing methods in terms of search recall, memory footprint, and re-indexing overhead for a vector database containing 50 million multi-modal vectors."
- "How do you detect, handle, and re-index vector collections when the underlying embedding model is upgraded to a higher-dimensional version?"
#### 4. LLM Evaluation, Telemetry & Performance Engineering
Evaluating non-deterministic generative systems in enterprise settings demands rigid, reproducible measurement methodologies. You will be tested on how you build automated test suites and production telemetry pipelines.
**Be ready to go over:**
- **Quantitative Metric Suites** – Implementing evaluation metrics using frameworks like Ragas or TruLens to track Context Precision, Context Recall, Faithfulness, and Answer Correctness.
- **LLM-as-a-Judge Design** – Building reliable synthetic evaluation datasets, managing judge bias, enforcing rubric-based evaluations, and calculating inter-annotator agreement score metrics.
- **Production Observability** – Instrumenting OpenTelemetry pipelines across LLM calls to capture prompt token usage, completion token counts, latency distributions ($P_{95}$, $P_{99}$), and function calling error rates.
**Example questions or scenarios:**
- "How would you design a Continuous Integration / Continuous Deployment (CI/CD) regression test suite that runs automatically whenever an engineer updates an agent's base system prompt?"
- "What key operational metrics and visualizations would you put on a dashboard for an engineering team managing an enterprise enterprise LLM fleet deployment?"
#### 5. Practical Hands-On Coding & Data Querying
**Salesforce** emphasizes real-world practical code over abstract algorithmic tricks. You are expected to demonstrate strong Python coding skills (data manipulation, asynchronous processing, standard design patterns) and advanced SQL querying capabilities.
**Be ready to go over:**
- **Data Processing & Scraping Pipelines** – Designing resilient, multi-threaded, or asynchronous Python services to process, clean, and format raw unstructured text streams.
- **SQL Data Analytics** – Writing complex SQL queries utilizing Common Table Expressions (CTEs), window functions (`ROW_NUMBER()`, `LEAD()`, `LAG()`), grouping sets, and aggressive filtering on large event datasets.
- **Prompt Engineering Task Implementation** – Constructing precise system prompts that meet specified enterprise functional requirements while strictly handling edge cases and constraint boundaries.
**Example questions or scenarios:**
- "Write a complete, runnable Python script that concurrently crawls a list of web endpoints, extracts main article content, sanitizes raw HTML, and queues batch payload tasks for an embedding service."
- "Write an advanced SQL query to compute the daily $P_{90}$ generation latency and total token cost per active tenant from an audit table of raw LLM interaction logs."
