Your question is Continuous vs Static Batching. Take a moment with it on the right.
Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).
Explain how continuous batching works and why it is superior to static batching for LLM serving.
Describe the request scheduler, token-level iteration, admission and removal of sequences, and the effect of variable generation lengths. Compare throughput, GPU utilization, latency, fairness, memory management, and implementation complexity. Discuss when static batching may still be preferable, and identify operational risks such as queue starvation, KV-cache exhaustion, and tail-latency regressions.