Project Context

You are the program owner for ShopSwift, a large e-commerce marketplace (40M monthly active buyers, ~1.8M daily orders) that recently expanded its in-house payments product, ShopSwift Pay, to reduce card processing fees and improve checkout conversion. Fraud losses have been rising, and the CFO has set a hard target to reduce fraud chargebacks by 20% QoQ without materially harming conversion.

A cross-functional team shipped a new real-time fraud detection model that scores every card-not-present transaction at checkout and either: (a) approves, (b) step-ups to 3DS / OTP, or (c) declines. The model is served via a low-latency feature store and a model-serving service in the critical payments path. The launch is high visibility because it directly impacts revenue, customer trust, and card network compliance. You launched to 10% traffic last week and ramped to 50% yesterday.

Team & Operating Model

What Happens (Model Failure Scenario)

Within 6 hours of the 50% ramp, dashboards show:

• Checkout conversion down 1.4 percentage points (baseline 71.0% → 69.6%) on scored traffic.
• False declines spike: customer support tickets for “payment failed” increase 35%.
• Fraud loss rate (early proxy using confirmed fraud + high-risk post-auth signals) is not improving; it appears slightly worse (+0.08% absolute).
• Latency p95 on the scoring endpoint increases from 45ms to 120ms, occasionally timing out, causing the payments service to fall back to a conservative rule that triggers step-up.

Risk Ops reports that many declined/step-up orders look like legitimate repeat buyers. ML engineers suspect feature drift: a key feature (“account_age_days”) is being computed incorrectly for users created via a new social login flow launched two weeks ago. Data Engineering notes that the streaming job that populates the feature store had a schema change merged by another team, and the model-serving service may be reading a default value.

Stakeholder Landscape (Competing Priorities)

• VP Payments: wants fraud reduction quickly; concerned about revenue impact and executive escalation.
• Head of Risk: wants immediate containment; prefers stricter controls to avoid chargeback thresholds with card networks.
• Growth PM (Checkout): wants conversion restored; argues the model is “breaking checkout” and should be rolled back now.
• Engineering Director (Payments Platform): wants stability and a clear incident process; worried about on-call load and cascading failures.
• Legal/Compliance: worried about adverse action explanations and regulatory scrutiny if legitimate customers are declined without clear rationale.

Constraints

1. SLA / Reliability: Payments authorization path must maintain 99.95% availability and p95 end-to-end checkout latency under 350ms.
2. Timeline: You have 48 hours before a major marketing campaign (expected +20% traffic) and 7 days before the monthly exec business review where this launch is a headline item.
3. Resource limits: No additional headcount; one ML engineer is on PTO for 3 days; Risk Ops is already at capacity.
4. Compliance: If chargeback rate exceeds 0.9% for the month, card network monitoring programs trigger, increasing fees and requiring remediation plans.
5. Technical: Model is integrated with feature store; changing features requires a backfill and validation. A full retrain takes 6–10 hours plus review.

Your Deliverables (What You Must Produce)

1. Incident response plan for the next 0–24 hours: triage, decision-making, communications, and immediate mitigations.
2. Rollback / containment strategy that balances fraud risk and conversion (including what traffic to roll back, and how you’ll monitor).
3. Root cause and corrective action plan: how you’ll confirm the failure mode (feature drift vs latency vs thresholding), fix it, and prevent recurrence.
4. Re-launch plan with gating criteria, ramp schedule, and cross-functional sign-offs.
5. Executive narrative: how you will explain what happened, what you traded off, and why your plan is the safest path.

Complications

1. Sales escalation: A top marketplace seller claims high-value orders are failing and threatens to pause ads spend unless fixed today.
2. Data dependency delay: The team that owns the upstream schema change says they can’t revert for 72 hours due to another launch.
3. Metric ambiguity: Fraud outcomes are delayed (chargebacks take days/weeks). You must rely on proxies without overreacting.

Your answer should show how you handle a model failure in production: how you decide whether to roll back, what you do in the first hours, how you coordinate stakeholders, and how you build a safer relaunch under tight constraints.