LLM Production Latency Handling

Describe a challenge you faced when deploying an LLM in production at Cisco, and how you handled latency.