Business Context

TaskRabbit’s marketplace matches clients (people who need help) with taskers (independent contractors) across categories like furniture assembly, moving, cleaning, and handyman work. The product team wants to improve the “Recommended Taskers” ranking shown on the task creation flow and search results.

Scale: TaskRabbit operates in ~100 metros, with ~2–4M monthly active clients, ~250K active taskers, and ~10–20M search/result impressions per day. A 1% relative lift in booking conversion is worth several million dollars annually, but bad recommendations can increase cancellations, refunds, and support costs.

You are asked to implement a recommendation/ranking system that, given a task request (category, time window, location, budget), returns a ranked list of taskers.

Dataset

You have 6 months of logged marketplace data. Each row represents a (request, tasker impression) pair from search results.

Size: ~120M impressions, ~35M unique requests, average 30 taskers per request.

Label characteristics:

• Booked rate per impression is low: ~1.5% (highly imbalanced).
• Position bias exists: higher-ranked taskers historically get more clicks/bookings.
• Cold start: ~18% of taskers have <5 ratings.

Success Criteria

1. Offline: Improve ranking quality on a held-out time period.
   • Target NDCG@10 ≥ 0.42 (baseline 0.38)
   • Target AUC-PR ≥ 0.12 (baseline 0.09) for booked prediction at impression level
2. Online (proxy): deliver a model that can plausibly drive +1–2% relative booking conversion without increasing cancellations.
3. Safety/quality: do not materially degrade post-booking outcomes.
   • Cancellation rate among booked jobs should not increase by > 0.2 pp.

Constraints

• Latency: P95 ranking under 120 ms per request (including feature fetch). You can precompute tasker features hourly/daily.
• Explainability: Ops/support wants a human-readable reason code (e.g., “high completion rate”, “fast response”).
• Fairness/market health: Avoid starving new taskers; include an exploration strategy.
• Data leakage: Features must be available at ranking time; no using future ratings or post-booking outcomes.

Deliverables (what you must produce)

1. Formulate the problem: ranking vs. classification; define the training label(s) and objective.
2. Propose a model approach that addresses position bias, class imbalance, and cold start.
3. Describe feature engineering and how you would build training examples without leakage.
4. Provide an evaluation plan: offline metrics, slicing (by category/metro/new taskers), and an online A/B test plan.
5. Provide a reference Python implementation for an offline training pipeline (data prep → model → evaluation) and how it would be deployed for low-latency serving.