Dataford
Popular roles
Software EngineerData AnalystData ScientistData EngineerBusiness AnalystAI EngineerMachine Learning EngineerProduct Manager
Browse
Browse All RolesEvery role hub, from analyst to MLBrowse All CompaniesCompany-specific interview loopsAll Interview GuidesThe full guide library
Top questions by role
Software EngineerData AnalystData ScientistData EngineerBusiness AnalystAI EngineerMachine Learning EngineerProduct Manager
Top questions by skill
SQLPythonStatisticsMachine LearningA/B TestingSystem DesignGenerative AIProduct SenseMetricsBehavioral
Browse all questions →Try a mock interview
ExperiencesCoaching
New
Practice
Mock InterviewsTimed interview simulations with feedbackSuccess PathYour 6-week structured planModulesCurated lessons by topicWebinarsTalks from ex-Big Tech data leadsPlaygroundA free-form scratch editor
Learn
BlogInterview strategy and career adviceTech Job Market ReportHiring trends across data and AI rolesFor UniversitiesDataford for career centersAbout DatafordWho we are and how we build
Pricing
Get started

Analyze New Feature Conversion Funnel

MediumMetrics00:00
Practice interviewer
In session
5 left
00:00

Whenever you're ready, walk me through how you'd approach this.

You need to log in / sign up to chat or submit.

Problem

Business Context

Notion has launched a new AI summary feature inside document pages. In the first 14 days after launch, product leadership wants to understand whether the feature is underperforming because users are not seeing it, not clicking it, or not completing the summary flow after clicking.

Metric Scenario

The feature is shown as a prominent in-product entry point on eligible document pages. Over the last 2 weeks, the team recorded 12.0M impressions, 720k clicks, and 86.4k completed summaries. This implies an overall CTR of 6.0%, post-click conversion rate of 12.0%, and impression-to-conversion rate of 0.72%. On desktop, CTR is 7.5% vs 3.8% on mobile. For new users, click-to-conversion is 8% vs 15% for existing users. The PM asks whether the main issue is top-of-funnel discovery, mid-funnel intent, or bottom-funnel completion friction.

Requirements

  1. Define the funnel metrics precisely from impression → click → conversion, including numerator, denominator, and event eligibility.
  2. Show how you would diagnose where the largest drop-off occurs and which stage should be prioritized.
  3. Propose a decomposition of the funnel by key segments to isolate likely drivers.
  4. Identify likely data quality pitfalls or instrumentation issues that could bias the analysis.
  5. Recommend 3 specific actions based on possible findings and name guardrail metrics to monitor.

Data Available

  • feature_impressions: impression_id, user_id, session_id, doc_id, timestamp, platform, country, user_tenure_days
  • feature_clicks: click_id, impression_id, user_id, session_id, timestamp, entry_point_location
  • feature_conversions: conversion_id, user_id, session_id, doc_id, timestamp, summary_type, latency_ms
  • user_profiles: user_id, workspace_id, plan_type, country, signup_date
  • page_context: doc_id, doc_type, doc_length_bucket, collaboration_state, page_views