Dataford
Interview QuestionsInterview GuidesExperiencesMock InterviewsPricing
Get started

Manage Production Model Drift

HardModel Evaluation00:00
Practice interviewer
In session
5 left
00:00

Your question is Manage Production Model Drift. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

You need to log in / sign up to chat or submit.

Problem

Scenario

You've shipped a model that was performing acceptably at launch, but over time its production behavior starts to change. The team wants a clear plan for detecting drift, understanding whether it is feature shift, label shift, or score drift, and deciding when to recalibrate, retune thresholds, retrain, or roll back.

Question

How would you manage model drift in a production AI system?

Representative Drift Signals

ECE·0.021 to 0.089AUC-ROC·0.93 to 0.86Positive rate·1.9% to 2.8%Recall at 0.62·0.74 to 0.61Top feature PSI·0.27Precision at 0.62·0.81 to 0.68