Compare Model Performance Across Datasets

Scenario

You are comparing the same model on multiple datasets, and the metrics do not line up the way you expected. One dataset looks strong, another is noticeably weaker, and a third sits in between. The team wants a clear read on whether the model is stable across data sources or if the results point to a real generalization problem.

Question

How do you evaluate the performance of a machine learning model across different datasets?