Business Context

Notion has launched a new AI summary feature inside document pages. In the first 14 days after launch, product leadership wants to understand whether the feature is underperforming because users are not seeing it, not clicking it, or not completing the summary flow after clicking.

Metric Scenario

The feature is shown as a prominent in-product entry point on eligible document pages. Over the last 2 weeks, the team recorded 12.0M impressions, 720k clicks, and 86.4k completed summaries. This implies an overall CTR of 6.0%, post-click conversion rate of 12.0%, and impression-to-conversion rate of 0.72%. On desktop, CTR is 7.5% vs 3.8% on mobile. For new users, click-to-conversion is 8% vs 15% for existing users. The PM asks whether the main issue is top-of-funnel discovery, mid-funnel intent, or bottom-funnel completion friction.

Requirements

1. Define the funnel metrics precisely from impression → click → conversion, including numerator, denominator, and event eligibility.
2. Show how you would diagnose where the largest drop-off occurs and which stage should be prioritized.
3. Propose a decomposition of the funnel by key segments to isolate likely drivers.
4. Identify likely data quality pitfalls or instrumentation issues that could bias the analysis.
5. Recommend 3 specific actions based on possible findings and name guardrail metrics to monitor.

Data Available

• feature_impressions: impression_id, user_id, session_id, doc_id, timestamp, platform, country, user_tenure_days
• feature_clicks: click_id, impression_id, user_id, session_id, timestamp, entry_point_location
• feature_conversions: conversion_id, user_id, session_id, doc_id, timestamp, summary_type, latency_ms
• user_profiles: user_id, workspace_id, plan_type, country, signup_date
• page_context: doc_id, doc_type, doc_length_bucket, collaboration_state, page_views