Rippling wants to measure whether an in-app campaign in the Rippling dashboard increases adoption of Rippling Payroll among eligible companies. Instead of sending the campaign to everyone, the growth team keeps a randomized holdout group with no exposure.
Use the holdout design to estimate incremental campaign lift and determine whether the observed difference in conversion is statistically significant.
The campaign targeted companies that had at least 25 employees, were using Rippling HRIS, and had not yet activated Rippling Payroll. Companies were randomly assigned at the company level for a 21-day test.
| Group | Companies | Payroll Activations | Conversion Rate |
|---|---|---|---|
| Holdout (no campaign) | 8,400 | 462 | 5.50% |
| Exposed (campaign shown in Rippling dashboard + email) | 25,200 | 1,575 | 6.25% |
Additional assumptions for interpretation:
| Parameter | Value |
|---|---|
| Significance level | 0.05 |
| Average annual gross profit per activated Payroll customer | $3,200 |
| Campaign delivery cost per exposed company | $4.50 |