Your question is Evaluating Multi-Agent Systems. Take a moment with it on the right.
Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).
What strategies would you use to evaluate the performance of a multi-agent system in a real-world application?
Discuss how you would evaluate both individual agents and the end-to-end workflow, including task success, reliability, safety, cost, latency, and user outcomes. Explain how you would combine offline test suites, simulation, human review, and online experimentation while accounting for non-determinism, delayed feedback, tool failures, distribution shift, and multi-agent interaction effects.
