Your question is LLM Production Evaluation Metrics. Take a moment with it on the right.
Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).
What metrics do you prioritize for LLM evaluation when moving from a prototype to a production environment?
Explain how your priorities change between offline validation and production monitoring. Address task quality, factuality, safety, latency, cost, reliability, and user experience, and describe how you would combine automated metrics with human evaluation and operational monitoring.