Dataford
Interview QuestionsInterview GuidesExperiencesMock InterviewsPricing
Get started

Network Interference in Collaboration Test

MediumA/B Testing & Experimentation00:00
I
Practice interviewer
Your interviewer
In session
I
Interviewer

Welcome to your interview.

The question is on your right: Network Interference in Collaboration Test. Take a moment with it first.

Talk your thinking through with me if you like - when you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes). Discussion and graded submissions share your five interviewer interactions, so spend them well.

You need to log in / sign up to chat or submit.

Problem

Scenario

You work on a collaborative design product and are testing a change to sharing or commenting behavior. Because users work in teams on the same files and projects, one user’s treatment experience could affect another user’s outcomes even if that second user is assigned to control. You are concerned the experiment may violate the usual independence assumptions.

Question

When would you worry about network interference in this experiment, and how could it distort the measured treatment effect? How would you redesign or analyze the test to account for that risk before deciding whether to ship?

What to Notice

  • Users collaborate in shared workspaces or projects
  • Treatment can change behavior of untreated collaborators
  • Naive user-level randomization may violate SUTVA
  • The right answer should discuss redesign, not just post-hoc analysis