Your question is Evaluate LLMs for Medical Text. Take a moment with it on the right.
Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).
How do you approach LLM evaluation when dealing with sensitive or specialized medical text?
Discuss how you would create representative evaluation data, protect patient and proprietary information, measure clinical correctness and hallucination, and validate model behavior without relying solely on automated judges. Address offline and online evaluation, expert review, refusal behavior, prompt injection, and the tradeoffs among accuracy, latency, and cost.