Holdout Lift for Payroll Campaign

Business Context

Rippling wants to measure whether an in-app campaign in the Rippling dashboard increases adoption of Rippling Payroll among eligible companies. Instead of sending the campaign to everyone, the growth team keeps a randomized holdout group with no exposure.

Problem Statement

Use the holdout design to estimate incremental campaign lift and determine whether the observed difference in conversion is statistically significant.

Given Data

The campaign targeted companies that had at least 25 employees, were using Rippling HRIS, and had not yet activated Rippling Payroll. Companies were randomly assigned at the company level for a 21-day test.

Additional assumptions for interpretation:

Requirements

1. State the null and alternative hypotheses for campaign lift.
2. Compute the observed absolute lift and relative lift versus holdout.
3. Run a two-proportion z-test using the randomized holdout as the control group.
4. Construct a 95% confidence interval for the absolute lift.
5. Estimate incremental activations caused by the campaign among exposed companies.
6. Estimate incremental gross profit net of campaign cost, and state whether you would scale the campaign.

Assumptions

• Randomization was done at the company level and exposure assignment was independent of baseline propensity.
• Each company is counted once, with no spillovers between companies.
• Payroll activation within 21 days is the primary outcome, and no major concurrent launches affected only one group.