Your question is Diagnose Novelty in UI Test. Take a moment with it on the right.
Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).
You work on a design software product team running an A/B test on a refreshed in-product navigation experience. Early treatment results look better on engagement, but you suspect users may simply be reacting to something new rather than finding it sustainably more useful.
What is the novelty effect, and why does it matter in product experiments? How would you tell whether the observed lift reflects lasting value versus a short-lived reaction to change?