Scenario

You are reviewing a change in user engagement after a product update. The team wants to know whether the observed shift is a real effect or just random variation in the data.

Question

What statistical methods would you use to decide whether a change in user engagement is real or just noise?