Project Context
You are the DPM (data program manager) for ShopSwift, a global e-commerce marketplace (~38M monthly active buyers, 2.4M daily orders). The company is rolling out a new executive KPI called “On-Time Delivery Rate (OTDR)” that will be used to (1) drive weekly operations reviews, (2) trigger carrier penalty clauses, and (3) inform a Q2 investor narrative about reliability improvements. OTDR will be published in a new Exec Metrics Hub dashboard and will be consumed by Finance for accruals.
The OTDR pipeline is built on top of events produced by the logistics platform (shipment created, out-for-delivery, delivered) and joined with promised delivery dates from the checkout service. The dataset is owned by a senior data scientist on the Logistics Analytics team (your colleague), who has been iterating quickly to hit a hard deadline: the dashboard must go live in 10 business days for the CEO’s QBR.
Two days into final validation, you notice an anomaly: OTDR for Brazil jumps from a stable ~86–88% to 99.6% starting last Monday, while customer support contacts about “late delivery” in Brazil have not decreased. At the same time, OTDR for Germany drops by 7 percentage points with no corresponding operational incident. Your colleague believes it’s “just a backfill artifact” and suggests shipping on time and cleaning it up later.
Stakeholder Landscape
- CEO Staff / BizOps: Wants a single, stable metric for QBR storytelling. Low tolerance for “it depends,” but high sensitivity to being embarrassed in front of the exec team.
- VP Logistics: Wants OTDR live to drive carrier accountability. Will push to launch even if some markets are imperfect, but cannot risk incorrect carrier penalties.
- Finance (RevRec + Accruals): Needs OTDR to be auditable; incorrect numbers could lead to material misstatement of carrier penalty accruals.
- Data Engineering: Owns the production pipelines and SLAs. They are already committed to a separate migration and can only spare limited on-call support.
- Privacy/Legal: Ensures event joins don’t inadvertently expose customer-level data in the exec layer; requires documented lineage and access controls.
You are accountable for execution: driving alignment, making trade-offs, and ensuring the launch is safe, credible, and on time.
Constraints
- Timeline: 10 business days to exec launch; CEO QBR date is immovable.
- Scope: OTDR must cover 12 priority markets (US, CA, MX, BR, UK, FR, DE, IT, ES, NL, IN, JP). Leadership expects a single global roll-up plus market drill-down.
- Resourcing:
- Data Engineering can provide 1 engineer at 25% for the next 2 weeks.
- Analytics Engineering has no additional headcount approved.
- Your colleague (dataset owner) is traveling for 3 days next week.
- Operational risk: Carrier penalty clauses can be triggered if OTDR falls below thresholds in certain lanes; wrong numbers could cause six-figure disputes.
- Technical reality: The pipeline includes a weekly backfill job, late-arriving events (up to 72 hours), and a new carrier integration in Brazil launched last week.
What You Need To Deliver (Candidate Tasks)
- Triage and investigation plan: How you will validate whether the anomaly is real vs. data defect, including who you involve and what checks you run.
- Decision framework: Clear criteria for whether to (a) block the launch, (b) launch with partial scope (e.g., exclude BR/DE), or (c) launch with known issues and disclosures.
- Execution plan and timeline: A day-by-day plan for the remaining 10 business days, including milestones, owners, and dependencies.
- Stakeholder communication: What you will tell the VP Logistics, Finance, and BizOps—especially if their preferences conflict.
- Launch safety plan: Monitoring, alerting, and a rollback/kill-switch approach if the metric is later found to be wrong.
Complications (Realistic Curveballs)
- Competing priority: The Engineering Director informs you that the data engineer supporting you is also on-call for a payments incident migration and may be pulled away with <24 hours notice.
- Scope pressure: BizOps asks for an additional breakout: OTDR by carrier in the top 5 markets, even though the carrier dimension is known to be messy in Brazil due to the new integration.
- Political tension: Your colleague feels accused and becomes defensive, arguing that “analytics always gets blamed” and that the exec team “won’t notice a single market spike.”
Your goal is to demonstrate how you would address a colleague’s dataset anomaly in a way that protects business decisions, maintains relationships, and still ships under a fixed executive deadline.
