A
You are a product analyst for a large knowledge-sharing platform that recently changed the home feed ranking to show more personalized answers and fewer followed-topic posts. In the first 2 weeks after launch, average weekly sessions per active user rose from 4.8 to 5.4 (+13%), answers read per session increased from 6.1 to 6.8 (+11%), and upvotes per weekly active user increased 7%. At the same time, early signs of retention are mixed: new-user 1-week retention is flat at 31%, while returning-user 4-week retention has moved from 58% to 56%. Leadership wants to know whether the change truly improved both near-term engagement and long-term retention, or whether the engagement lift is masking lower habit formation.
How would you evaluate whether this change improved both engagement and long-term retention, and how would you distinguish a real durable improvement from a short-term lift that could hurt user value over time?
