Whenever you're ready, walk me through how you'd approach this.
How do you evaluate the performance of generative models or large language models in production settings?
Discuss how you would combine offline evaluation, human review, automated model-based graders, online experiments, safety monitoring, latency, cost, and distribution-shift detection. Explain how you would evaluate open-ended outputs when there is no single correct answer, define representative test slices, and detect regressions after model or prompt changes. Include a practical implementation that computes relevant metrics from prediction logs and reports uncertainty.