Business Context
StreamCart, a subscription video platform, changed its landing-page signup flow and wants to know whether the observed conversion increase is a real effect or just random variation.
Problem Statement
You are given results from a 10-day A/B test. Determine whether the treatment's higher conversion rate is statistically significant, quantify the uncertainty, and explain whether the team should roll out the redesign.
Given Data
| Group | Users Exposed | Conversions | Conversion Rate |
|---|
| Control (old signup flow) | 18,420 | 2,211 | 12.00% |
| Treatment (new signup flow) | 18,105 | 2,316 | 12.79% |
Additional test settings:
| Parameter | Value |
|---|
| Significance level | 0.05 |
| Test type | Two-sided |
Requirements
- State the null and alternative hypotheses for the conversion-rate change.
- Compute the sample conversion rates and the observed lift.
- Use a two-proportion z-test to calculate the pooled proportion, standard error, z-statistic, and p-value.
- Construct a 95% confidence interval for the difference in conversion rates.
- Decide whether the observed change is likely real or just noise at the 5% level.
- Explain the result in business terms, including whether the lift is practically meaningful.
Assumptions
- Users were randomly assigned to control and treatment.
- Each user appears once and conversion is binary.
- The normal approximation is appropriate because both groups have large sample sizes and enough successes/failures.
- No major instrumentation issues or sample-ratio mismatch occurred during the experiment.
