Dataford
Interview QuestionsInterview GuidesExperiencesMock InterviewsPricing
Get started

Evaluate Models Across Datasets

MediumModel Evaluation00:00
Practice interviewer
In session
5 left
00:00

Your question is Evaluate Models Across Datasets. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

You need to log in / sign up to chat or submit.

Problem

Scenario

You have trained a model and now need to compare its performance across multiple datasets, such as training, validation, test, and a newly collected holdout set. The team wants to know whether the model is truly generalizing or if performance differences are coming from dataset shift, class imbalance, or inconsistent score calibration.

Question

How do you evaluate the performance of a machine learning model across different datasets?

Observed Metrics

Test AUC·0.89Holdout F1·0.66Holdout AUC·0.82Training F1·0.84Training AUC·0.95Validation AUC·0.90