Welcome to your interview.
The question is on your right: Evaluating Agent Performance Beyond Matching. Take a moment with it first.
Talk your thinking through with me if you like - when you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes). Discussion and graded submissions share your five interviewer interactions, so spend them well.
You are evaluating an AI agent that retrieves context, uses tools, and produces multi-step answers. Exact string matching is clearly too narrow because the agent can succeed with different wording, partially fail in retrieval, or take an inefficient tool path while still sounding correct.
How do you evaluate the performance of an AI agent beyond simple output matching (e.g., using RAGAs or custom evaluation frameworks)?