Whenever you're ready, walk me through how you'd approach this.
You are designing the serving stack for a product that uses large language models in production. Different requests have different latency, cost, and safety needs, and the system must handle traffic from multiple user segments.
How would you design an LLM serving system that balances latency, cost, scalability, and safety?