Business Context

You’re on the ML Platform team at MercuryMart, a global e-commerce marketplace with 35M monthly active users and ~120M active SKUs. A new regulatory policy in the EU requires the company to correctly classify products into a restricted set of 42 compliance categories (e.g., medical devices, children’s products, lithium batteries) to drive downstream workflows: listing eligibility, shipping restrictions, and mandatory disclosures.

Misclassification has real cost: false negatives can trigger regulatory fines and forced delistings, while false positives reduce seller revenue and harm trust. The business wants a model in production within 8 weeks, and the system must support near-real-time classification for newly created listings.

The key decision: use a pre-trained model (transfer learning) vs train a model from scratch. You must propose and justify an approach, including a plan to evaluate trade-offs and deploy safely.

Dataset

Each listing has a title, description, and structured metadata. Labels come from a combination of seller-provided category + human review; label noise is non-trivial.

• Size: ~8.5M labeled listings over 18 months
• Target: 42-way multiclass compliance category
• Class balance: long tail — top 5 classes = 62% of data; bottom 10 classes each <0.3%
• Missing data: 20% missing numeric shipping attributes; 40% missing images; 8% missing descriptions
• Label noise: estimated 3–7% due to seller mislabeling and policy changes

Success Criteria

1. Macro-F1 ≥ 0.72 on a time-based holdout (last 6 weeks)
2. Recall ≥ 0.90 for the 6 “high-risk” categories (regulated items), measured per-class
3. p95 online inference latency ≤ 120 ms per listing (text+metadata only; images are best-effort)
4. Must provide auditable explanations for compliance reviewers (at least at the feature/token level)

Constraints

• Training budget: up to 4× A100 GPUs for 48 hours (or equivalent)
• Serving: CPU-only is preferred; GPU serving is allowed only if it materially improves high-risk recall
• Must handle multilingual inputs and taxonomy drift (new brands, new seller behaviors)
• Model refresh: at least weekly, with a rollback plan

Deliverables (what you must produce in the interview)

1. A recommendation: pre-trained vs from-scratch, or a hybrid, with a clear decision framework.
2. A proposed modeling approach (architecture + features) and how you’ll handle imbalance, missingness, and label noise.
3. An evaluation plan: splits, metrics, and how you’ll tune thresholds for high-risk classes.
4. A production plan: latency strategy, monitoring, retraining cadence, and safe rollout.
5. A brief risk assessment: failure modes and mitigations (policy changes, adversarial sellers, drift).