You work on a B2B SaaS product that sells IT management software to mid-market companies. The growth team wants to test a new paid social acquisition channel that targets IT admins and operations leaders, and believes the channel may bring higher-intent traffic than existing paid search but at a higher top-of-funnel cost. You can route eligible ad traffic to either the new channel treatment or the current paid acquisition mix as control, then measure downstream product and sales outcomes. The team wants evidence that the new channel improves efficient pipeline generation, not just clicks or form fills.
How would you design this experiment end to end so you can decide whether to scale the new paid social channel, including the success metric, randomization approach, power and MDE, analysis plan, and how you would protect against common experimentation pitfalls?