Your question is Rare Event Detection Under Imbalance. Take a moment with it on the right.
Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).
You're evaluating a classifier for rare system anomalies or rider complaints, where positives are much less common than negatives. You need to decide how to assess model quality and how to handle the imbalance without misleading yourself.
How do you handle highly imbalanced datasets when trying to predict rare system anomalies or rider complaints?