Your question is LLM Metrics for Customer Chatbots. Take a moment with it on the right.
Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).
What metrics would you use for LLM evaluation when deploying a customer-facing chatbot?
Discuss how you would evaluate quality before launch and monitor it in production. Cover task success, factuality and hallucination, safety, user experience, latency, reliability, and cost. Explain which metrics require human or LLM-based judgment, how you would construct a representative test set, and how you would set thresholds or investigate regressions.