Business Context
StreamCart, a subscription video platform, tested a new signup page intended to improve trial-start conversion. Traffic was randomly split between the current page (control) and the redesigned page (treatment) for 10 days.
Problem Statement
Determine whether the observed change in conversion rate is statistically significant, and quantify the likely size of the true lift.
Given Data
| Group | Sample Size | Conversions | Conversion Rate |
|---|
| Control | 18,420 | 2,045 | 11.10% |
| Treatment | 18,105 | 2,146 | 11.85% |
Use a two-sided hypothesis test with significance level α=0.05.
Requirements
- State the null and alternative hypotheses for the conversion rates.
- Compute the sample conversion rates and the observed absolute lift.
- Calculate the pooled proportion and pooled standard error for a two-proportion z-test.
- Compute the z-statistic and two-sided p-value.
- Construct a 95% confidence interval for the difference in conversion rates (ptreatment−pcontrol).
- Decide whether the redesign should be considered statistically significant at the 5% level.
- Briefly explain whether the result is also practically meaningful for the business.
Assumptions
- Users were independently and randomly assigned to control or treatment.
- Each user appears once in the analysis.
- The conversion event is binary: either a user starts a trial or does not.
- Sample sizes are large enough for the normal approximation to be appropriate.
