Business Context

You’re on the perception team at MetroDrive, a ride-hailing + autonomous delivery company operating in 6 major US cities. Your vehicles run an on-device camera model that classifies nearby vehicles into categories used by downstream planning: {car, truck, bus, motorcycle, bicycle, emergency_vehicle}. Missing an emergency vehicle (ambulance, fire truck, police) is a safety-critical failure: the planner may not yield or may choose an unsafe maneuver. However, emergency vehicles are rare in the training data (a classic long-tail problem), and the model “rarely sees” them during training.

The current model performs well on common classes but has poor recall on emergency_vehicle, especially at night, in rain, and when lights are partially occluded.

Dataset

You have a curated dataset from the last 90 days of fleet driving.

Success Criteria

Your goal is to improve performance on the long-tail class without breaking overall system behavior.

1. Emergency vehicle recall ≥ 0.85 at precision ≥ 0.60 on a held-out test set representing the most recent 2 weeks.
2. Overall macro F1 ≥ 0.78 (to avoid collapsing other classes).
3. Night + rain slice: emergency recall ≥ 0.75.
4. No more than +10% increase in average inference latency vs the current baseline model.

Constraints

• You cannot collect new labeled data immediately; you can only use the existing dataset plus unlabeled fleet video.
• The model must be robust across cities and camera hardware revisions.
• You must propose an approach that can be implemented and iterated in production (monitoring + retraining plan).

Deliverables

1. Propose a modeling and training strategy to address the long-tail emergency vehicle class (data, loss, sampling, augmentation, architecture choices).
2. Define a validation strategy that avoids leakage and gives reliable estimates for rare-class performance.
3. Describe how you would use unlabeled data (if at all) to improve emergency-vehicle performance.
4. Provide an evaluation plan (metrics + slice metrics + thresholding) and how you’d pick an operating point.
5. Outline a production plan: monitoring, drift detection, and how you’d trigger retraining for the tail class.