Welcome to your interview.
The question is on your right: Diagnose Sudden Accuracy Drop. Take a moment with it first.
Talk your thinking through with me if you like - when you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes). Discussion and graded submissions share your five interviewer interactions, so spend them well.
You've deployed a model and the team sees a sudden drop in production performance after a period of stable results. Offline training and validation looked healthy before launch, but recent monitoring suggests the model is making worse decisions in production. You need to figure out whether this is a data issue, a serving issue, a threshold or calibration problem, or a real change in the population the model is scoring.
If a deployed model's accuracy drops suddenly, what steps do you take to diagnose and fix the issue?