Checkout Button Color A/B Significance

Business Context

CartLane, an e-commerce retailer, changed the checkout button from gray to green and ran a randomized A/B test to see whether the new color increased completed purchases.

Problem Statement

Determine whether the new checkout button color caused a statistically significant increase in sales conversion rate, and quantify the size of the effect.

Given Data

The experiment ran for 14 days at the user level. Each user saw only one version.

Additional parameters:

Requirements

1. State the null and alternative hypotheses.
2. Compute the sample conversion rates and the absolute lift.
3. Use a two-proportion z-test to test for statistical significance.
4. Calculate the p-value.
5. Construct a 95% confidence interval for the difference in conversion rates.
6. Conclude whether the button color change should be rolled out based on both statistical and practical significance.

Assumptions

• Random assignment was implemented correctly.
• Users are independent observations.
• Conversion is binary: each exposed user either purchased or did not purchase.
• No major concurrent site changes affected checkout behavior during the test window.
• Sample sizes are large enough for normal approximation to the binomial distribution.