Your question is Fraud ML Model Evaluation Strategy. Take a moment with it on the right.
Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).
Fraud model evaluation: train and release the next Fraud ML model. Discuss model updates, training data, attribute monitoring, how to define metrics (AUC, ROC, PR, F1), handling the imbalanced dataset, PR vs ROC, and how to pick thresholds.
Asked in the technical phone screen stage. First of two case studies from a senior MLE; second case study is points-walled. Reported follow-ups: PR vs ROC, which is more sensitive with imbalanced data? How to pick thresholds.