Context

RideNow uses a gradient-boosted regression model to predict next-day driver demand by city zone and hour so operations can position incentives and staffing. Over the last 6 weeks, business leaders reported that predicted demand has become materially different from actual completed rides, causing overspending in some zones and driver shortages in others.

Current Performance

The Problem

The model is systematically overpredicting demand in suburban commuter zones and underpredicting demand near airports and event venues. Operations estimates that forecast error increased weekly incentive waste by $180K and reduced fulfilled ride rate by 2.7 percentage points during peak periods.

Requirements

1. Diagnose the most likely reasons predictions are materially different from actual outcomes.
2. Interpret what the metric pattern says about model quality versus calibration versus data drift.
3. Propose a structured error analysis plan across time, geography, and demand segments.
4. Recommend specific model, feature, and threshold/policy changes to reduce business impact.
5. Explain what you would monitor going forward and what rollback or fallback policy you would use.

Constraints

• Incentive budget cannot increase by more than 5%.
• Retraining takes 10 days end-to-end.
• Peak-hour underprediction is more costly than off-peak overprediction.
• City launches and major events can change demand patterns within days.