You observe that ad performance on a rider-facing ads surface dropped after a product change. In the 4 weeks before the change, there were 8,400,000 impressions and 100,800 clicks; in the 4 weeks after, there were 8,900,000 impressions and 89,000 clicks. You also have weekly data split by advertiser segment: local advertisers had CTR move from 1.50% to 1.20% while their impression share rose from 40% to 55%, and national advertisers had CTR move from 0.90% to 0.80% while their impression share fell from 60% to 45%. A comparable untreated surface showed CTR move from 1.05% to 1.02% over the same periods.
How would you determine whether the observed drop is primarily explained by the product change, normal seasonality, or a shift in advertiser mix, and what does the evidence suggest here?