Top 50 model inference Interview Questions

Determine whether model inference is memory bound or compute bound, then choose profiling evidence and optimizations.

gpu hardwareinference latencyml inferenceApple

Explain why transformer inference is often memory-bound on modern GPUs and how to reduce bandwidth pressure.

gpu hardwaremodel inferencecomputational costAdobe

Design a serving layer that supports greedy, beam, top-k, and top-p decoding with low latency and safe fallbacks.

production systemsinference latencymodel inferenceMicrosoft

Explain LLM fine-tuning with LoRA, focusing on low-rank adapters, rank, alpha, serving, and evaluation tradeoffs.

gpu hardwaremodel inferencecomputational costAmazon

Explain how token IDs become dense vectors and how those embeddings are trained and served in an LLM.

gpu hardwareml inferencemodel inferenceAmazon

Explain attention mechanics and how you would operationalize them in a production agent model.

agentic systemsmodel inferenceagent designAmazon