Context

StreamLine runs a personalized home feed that ranks posts, videos, and articles for each user session. A new recommendation model increased click-through rate after launch, but product leaders now see weaker downstream engagement and more user complaints about repetitive content.

Current Performance

The Problem

The ranking model appears better on top-line click metrics, but worse on satisfaction, diversity, and retention. You need to determine whether the model is genuinely better, over-optimized for clicks, or poorly calibrated in production.

Requirements

1. Interpret the metric tradeoffs and identify the most important warning signals.
2. Explain why CTR and offline ranking gains may still lead to a worse user experience.
3. Recommend an evaluation framework covering offline metrics, online metrics, and segment-level analysis.
4. Propose 3-5 concrete model or objective changes to improve feed quality.
5. Discuss how you would validate improvements with an online experiment.

Constraints

• StreamLine optimizes for long-term engagement, not clicks alone.
• The feed must balance relevance, freshness, and creator diversity.
• Product policy limits any change that reduces 7-day retention by more than 0.5 percentage points during testing.