Business Context

NotionHub, a team collaboration SaaS product, launched a new AI meeting summary feature to 20% of its workspace customers four weeks ago. Product leadership wants a clear metric framework before expanding the rollout to all users.

Metric Scenario

The feature is available to 12,000 weekly active users across 1,800 workspaces. In the last 4 weeks, 4,800 users viewed the feature entry point, 2,100 generated at least one summary, 1,050 generated summaries in 2 or more separate weeks, and 420 shared a summary with teammates. Weekly retention among users who generated a first summary is 38% in week 2, compared with 52% retention for the broader core product. PMs believe the feature could improve collaboration and reduce meeting follow-up time, but Finance is concerned about inference cost per summary and Support has reported a small increase in complaint tickets about inaccurate summaries.

Requirements

1. Define the primary success metric for this feature and explain why it should be the north star.
2. Identify 3-5 supporting metrics across adoption, engagement, retention, and quality.
3. Separate leading indicators from lagging indicators.
4. Propose guardrail metrics to ensure the feature does not harm the core product or unit economics.
5. Explain how you would decide whether the feature is successful enough for full rollout.

Data Available

• feature_impressions: user_id, workspace_id, timestamp, entry_point, device_type
• summary_generation_events: user_id, workspace_id, meeting_id, timestamp, summary_length, generation_status
• summary_share_events: user_id, workspace_id, meeting_id, shared_to_count, timestamp
• workspace_activity_daily: workspace_id, active_users, meetings_created, comments_posted, docs_created, date
• support_tickets: ticket_id, workspace_id, user_id, category, timestamp
• ai_cost_logs: workspace_id, meeting_id, tokens_used, inference_cost, timestamp