You are analyzing an online experiment for a streaming product after a new discovery or playback experience was tested. Early results look promising, but you know online experiments can be distorted by issues like short-term excitement, users affecting each other, or instrumentation problems.
What pitfalls would you watch for when analyzing this experiment, such as novelty effects or network interference, and how would each one bias the results or change your interpretation?