Business Context
StreamCart, a SaaS company, tested a redesigned pricing page intended to increase free-trial signups. The product manager wants a statistically sound recommendation on whether to launch the new page.
Problem Statement
You are given results from a 21-day randomized A/B test. Determine whether the new pricing page produced a statistically significant improvement in signup conversion rate versus the current page.
Given Data
| Group | Sample Size | Signups | Conversion Rate |
|---|
| Control (current page) | 18,400 | 2,208 | 12.0% |
| Treatment (new page) | 17,900 | 2,327 | 13.0% |
Additional test settings:
| Parameter | Value |
|---|
| Significance level | 0.05 |
| Test type | Two-tailed |
Requirements
- State the null and alternative hypotheses.
- Compute the sample conversion rates for both groups.
- Calculate the pooled proportion and pooled standard error for a two-proportion z-test.
- Compute the z-statistic and two-tailed p-value.
- Construct a 95% confidence interval for the difference in conversion rates.
- Decide whether the result is statistically significant at the 5% level.
- Explain whether the observed lift is large enough to matter from a business perspective.
Assumptions
- Users were randomly assigned to control and treatment.
- Each observation represents a unique user.
- Independence between users is reasonable.
- Normal approximation is appropriate because both groups have large sample sizes and sufficient numbers of successes and failures.
