Your question is Evaluate a Prompt Change. Take a moment with it on the right.
Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).
You maintain an LLM-powered feature and want to update the system prompt. You have a fixed set of 1,000 diverse user queries that represent real usage, and you need a reliable way to tell whether the new prompt is actually better or if it regresses performance on important cases.
How would you design a test suite to evaluate whether a prompt change improved or degraded model performance across 1,000 diverse user queries?