Business Context

Zillow has launched a new Saved Search Alerts feature in its mobile app. Users can save a home search (filters like price, beds, neighborhood) and receive notifications when new listings match. Four weeks after launch, product leadership wants to know whether the feature is actually successful, not just whether people clicked on it.

The launch reached 2.4M monthly active app users. In the first 28 days, 420K users viewed the save-search prompt, 126K created at least one saved search, 88K enabled notifications, and 31K returned to a listing from an alert. Among users who created a saved search, 7-day retention was 41% versus 34% for comparable home-search users who did not. However, unsubscribe rate from alerts reached 18% within 14 days, and app store complaint volume about notifications increased 9% week over week.

Requirements

1. Define the primary success metric for the feature and explain why it should be the north-star metric for this launch.
2. Propose a set of supporting metrics across adoption, engagement, retention, conversion, and guardrails.
3. Show how you would decompose feature performance to determine whether the feature is creating real user value or just generating notification noise.
4. Identify the most important segments to compare and the likely reasons performance may differ across them.
5. Recommend what threshold would make you call the launch a success after the first month.

Data Available

• app_events: feature impressions, save-search creation, edit, delete, notification opt-in, alert click, listing view
• user_sessions: session timestamps, platform, app version, session length
• listings_funnel: listing view, contact agent, schedule tour, apply, purchase intent events
• user_profiles: market, buyer/renter flag, account age, notification settings
• push_logs: notification sent, delivered, opened, muted, unsubscribed