A/B Test for Rippling Landing Page

Business Context

Rippling's growth team wants to test a new hero message on the main payroll landing page to improve visitor-to-demo conversion. Traffic is split 50/50 between the current page in control and the new message in treatment.

Problem Statement

Use the experiment results below to determine whether the new landing page message increased demo request conversion rate. Assume this is the primary metric and the team will make a rollout decision based on statistical evidence at the 5% significance level.

Given Data

Additional test settings:

Requirements

1. State the null and alternative hypotheses for the conversion rate difference.
2. Compute the sample conversion rates and the absolute lift.
3. Calculate the pooled proportion and the standard error for a two-proportion z-test.
4. Compute the z-statistic and p-value.
5. Construct a 95% confidence interval for the treatment minus control conversion lift.
6. Decide whether Rippling should roll out the new message based on both statistical and practical significance.

Assumptions

• Visitors were randomly assigned and each visitor appears once.
• No major traffic source mix shift occurred during the test window.
• The normal approximation is valid because both groups have large sample sizes and sufficient conversions/non-conversions.