Business Context

You’re a senior data scientist at StreamCart, a large e-commerce marketplace (~8M weekly active users, ~$40M/week in gross merchandise value). Marketing spend is split across Paid Search, Paid Social, and Email, and leadership wants a defensible answer to: “Which channels actually cause incremental purchases?”

The challenge: user journeys are multi-touch and highly confounded. For example, high-intent users are more likely to click Paid Search and also more likely to purchase even without ads. The growth team proposes a multi-touch attribution (MTA) model using observational data, but Finance is demanding a causal interpretation and uncertainty estimates.

To reduce confounding, the team ran a geo-level holdout: 80 DMAs (geographies) were randomly assigned for 4 weeks to either keep Paid Social at baseline or increase Paid Social spend by ~25%. Other channels continued as usual. You will use this experiment to (a) estimate the incremental effect of Paid Social and (b) translate that into a data-driven attribution weight relative to other channels.

Given Data

Outcome is weekly purchases per 10,000 active users in each DMA-week.

Assume each DMA-week observation is approximately independent (you can critique this later).

Problem Statement

1. Quantify the incremental lift in purchases attributable to the Paid Social spend increase using a statistically valid method and uncertainty.
2. Explain how you would incorporate this result into a multi-touch attribution system so that channel credit reflects incrementality rather than correlation.

Requirements

1. Define the estimand for incrementality (e.g., ATE on purchases/10k) and state H₀/H₁.
2. Compute the difference in means, its standard error, a 95% confidence interval, and a p-value.
3. Translate the lift into an incremental purchases per additional $1,000 of Paid Social spend. (Use the fact that treatment increased Paid Social spend by 25% from a baseline of $12,000 per 10k users/week.)
4. Propose a practical MTA approach that uses this experiment to calibrate observational attribution (e.g., constrained regression / Bayesian prior / scaling heuristic). Be explicit about what you would change in the model and why.
5. List at least 3 caveats (interference, time effects, measurement, multiple channels moving, etc.) and how you’d mitigate them.

Assumptions and Constraints

• Randomization is at the DMA level; within a DMA, users are exposed to a mix of channels.
• Purchases are approximately normally distributed at the DMA-week aggregation scale (CLT).
• No major product changes occurred during the 4-week test.
• You may treat the two groups as independent samples for the core calculation, but you should discuss whether clustering by DMA matters.