Conversion Lift Significance for Signup Redesign

Business Context

StreamCart, a subscription video platform, changed its landing-page signup flow and wants to know whether the observed conversion increase is a real effect or just random variation.

Problem Statement

You are given results from a 10-day A/B test. Determine whether the treatment's higher conversion rate is statistically significant, quantify the uncertainty, and explain whether the team should roll out the redesign.

Given Data

Additional test settings:

Requirements

1. State the null and alternative hypotheses for the conversion-rate change.
2. Compute the sample conversion rates and the observed lift.
3. Use a two-proportion z-test to calculate the pooled proportion, standard error, z-statistic, and p-value.
4. Construct a 95% confidence interval for the difference in conversion rates.
5. Decide whether the observed change is likely real or just noise at the 5% level.
6. Explain the result in business terms, including whether the lift is practically meaningful.

Assumptions

• Users were randomly assigned to control and treatment.
• Each user appears once and conversion is binary.
• The normal approximation is appropriate because both groups have large sample sizes and enough successes/failures.
• No major instrumentation issues or sample-ratio mismatch occurred during the experiment.