Business Context
BrightCart ran an email campaign offering free shipping to a randomly selected group of users. The marketing team wants to know whether the observed lift in purchase conversion is real or just sampling noise.
Problem Statement
Use the campaign test results to determine whether the campaign's lift in conversion rate is statistically significant.
Given Data
| Group | Sample Size | Purchases | Conversion Rate |
|---|
| Control (no email) | 18,500 | 1,554 | 8.40% |
| Treatment (email sent) | 17,900 | 1,665 | 9.30% |
Additional parameters:
| Parameter | Value |
|---|
| Significance level | 0.05 |
| Test type | Two-tailed |
| |
| The observed lift is 0.93% - 0.84% = 0.009, or 0.9 percentage points. | |
Requirements
- State the null and alternative hypotheses for the campaign lift.
- Compute the sample conversion rates and the pooled proportion.
- Calculate the standard error for a two-proportion z-test.
- Compute the z-statistic and p-value.
- Construct a 95% confidence interval for the lift in conversion rate.
- Decide whether the lift is statistically significant at the 5% level.
- Briefly explain whether the result is also practically meaningful for the business.
Assumptions
- Users were randomly assigned to treatment and control.
- Each user contributes at most one purchase outcome.
- The normal approximation is valid because both groups have large sample sizes and enough conversions/non-conversions.
- No major targeting or delivery bias affected which users received the campaign.
