Your question is Cost-Efficient LLM Serving. Take a moment with it on the right.
Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).
Describe the key considerations when designing a system for LLM serving that prioritizes both cost efficiency and user experience.
Discuss how you would clarify workload requirements, select and route models, manage latency and GPU utilization, and design online and batch components. Cover evaluation, observability, failure handling, and training-serving consistency where relevant.