You’re a senior data scientist at StreamCart, a large e-commerce marketplace (~8M weekly active users, ~$40M/week in gross merchandise value). Marketing spend is split across Paid Search, Paid Social, and Email, and leadership wants a defensible answer to: “Which channels actually cause incremental purchases?”
The challenge: user journeys are multi-touch and highly confounded. For example, high-intent users are more likely to click Paid Search and also more likely to purchase even without ads. The growth team proposes a multi-touch attribution (MTA) model using observational data, but Finance is demanding a causal interpretation and uncertainty estimates.
To reduce confounding, the team ran a geo-level holdout: 80 DMAs (geographies) were randomly assigned for 4 weeks to either keep Paid Social at baseline or increase Paid Social spend by ~25%. Other channels continued as usual. You will use this experiment to (a) estimate the incremental effect of Paid Social and (b) translate that into a data-driven attribution weight relative to other channels.
Outcome is weekly purchases per 10,000 active users in each DMA-week.
| Item | Value |
|---|---|
| DMAs | 80 |
| Weeks | 4 |
| Total observations | 320 |
| Treatment DMAs (Paid Social +25%) | 40 |
| Control DMAs | 40 |
| Mean purchases/10k (Control) | 312.4 |
| Mean purchases/10k (Treatment) | 327.9 |
| SD of purchases/10k (Control, across DMA-week obs) | 44.8 |
| SD of purchases/10k (Treatment, across DMA-week obs) | 46.1 |
| Mean Paid Search spend ($/10k users/week) | 18,200 |
| Mean Email sends (per 10k users/week) | 41,000 |
| Correlation between Paid Search spend and purchases | 0.62 |
| Significance level | α = 0.05 |
Assume each DMA-week observation is approximately independent (you can critique this later).