Business Context
StreamCart, a subscription video platform, tested a new signup page against the current page to improve trial-start conversion. Users were randomly split between control and treatment for one week.
Problem Statement
Use hypothesis testing to determine whether the new signup page changed conversion rate relative to the current page.
Given Data
| Group | Sample Size | Trial Starts | Conversion Rate |
|---|
| Control (current page) | 18,500 | 2,220 | 12.00% |
| Treatment (new page) | 17,900 | 2,327 | 13.00% |
Use a two-sided test at significance level α = 0.05.
Requirements
- State the null and alternative hypotheses.
- Compute the sample conversion rates for both groups.
- Calculate the pooled proportion under the null hypothesis.
- Compute the standard error for the difference in proportions.
- Calculate the z-statistic and the two-sided p-value.
- Decide whether the result is statistically significant at the 5% level.
- Briefly explain what the result means for a rollout decision.
Assumptions
- Random assignment was implemented correctly.
- Each observation is an independent user.
- The normal approximation is appropriate because both groups have large sample sizes and enough conversions/non-conversions.
- No major instrumentation issues or traffic-quality differences occurred during the experiment.
