Business Context

AcmeShop, a mobile commerce app, launched a new promotional message: "Get 20% off your first order this week". The growth team sent it to 2.4M users across push notifications, email, and in-app banners. One week later, leadership wants to know whether the message is actually resonating with users or simply generating superficial opens.

Metric Scenario

Before the launch, AcmeShop's weekly baseline was: push open rate 8.5%, email open rate 24%, click-through rate 3.2%, first-order conversion rate among exposed users 1.8%, and 14-day repeat purchase rate for first-time buyers 22%. After launch, push open rate rose to 10.1%, email open rate to 27%, CTR to 4.6%, and first-order conversion among exposed users to 2.3%. However, unsubscribe rate increased from 0.18% to 0.34%, and 14-day repeat purchase for newly converted users is still unknown because only 7 days have passed.

The VP of Growth asks whether the new message is resonating, which user segments respond best, and what metric framework should be used so the team does not over-index on vanity metrics.

Requirements

1. Define the primary metric you would use to judge whether the message or offer is resonating.
2. Identify leading indicators and lagging indicators that should be tracked together.
3. Decompose performance by funnel stage and user segment to diagnose where resonance is strongest or weakest.
4. Explain how you would distinguish genuine resonance from short-term discount harvesting.
5. Recommend what AcmeShop should do next if top-of-funnel engagement improves but downstream retention worsens.

Data Available

• message_delivery_log: send_time, channel, campaign_id, user_id, delivered_flag
• message_engagement_log: opens, clicks, dismissals, unsubscribes by user and campaign
• orders: order_id, user_id, order_time, gross_merchandise_value, discount_amount, is_first_order
• user_profile: signup_date, acquisition_channel, country, device_type, loyalty_status
• retention_events: repeat_order_flag, repeat_order_time, 7d/14d/30d activity