Whenever you're ready, walk me through how you'd approach this.
Design a system for LLM serving that needs to handle high concurrency while maintaining low latency. What caching strategies would you employ?
Discuss the serving architecture, cache layers, cache keys and invalidation, request coalescing, model and prompt versioning, privacy, consistency, capacity management, and fallbacks. Treat latency, cost, cache effectiveness, training-serving skew, and production monitoring as first-class concerns.