You've deployed a model and the team sees a sudden drop in production performance after a period of stable results. Offline training and validation looked healthy before launch, but recent monitoring suggests the model is making worse decisions in production. You need to figure out whether this is a data issue, a serving issue, a threshold or calibration problem, or a real change in the population the model is scoring.
If a deployed model's accuracy drops suddenly, what steps do you take to diagnose and fix the issue?
Interpret whether the drop is threshold-related or ranking-relatedUse confusion matrix changes to reason about business impactCheck calibration drift and feature driftPrioritize diagnosis before choosing retrain, recalibrate, or rollback