You work on a work-management product and your team has built a new AI-generated nudge in the notifications and inbox experience that suggests teammates clarify task ownership when a project appears stalled. The team believes the nudge will increase weekly active collaboration, but it may also create spillovers because one person’s message can change behavior for everyone else on the same project. A standard user-level A/B test may violate independence if treated and control users collaborate in the same workspace. You need to decide whether to run a switchback test and how to design it.
How would you design this experiment, including whether a switchback test is more appropriate than a standard A/B test, what you would use as the primary metric and guardrails, how you would power it, and how you would analyze and interpret the results given interference risk?