Business Context
StreamCart, an online marketplace, tested a new promotional email subject line to improve click-through rate (CTR). The marketing team wants to know whether the observed lift is real or just random variation.
Problem Statement
Use a hypothesis test to determine whether the new subject line produced a statistically significant change in CTR versus the current subject line.
Given Data
| Group | Emails Sent | Clicks | CTR |
|---|
| Control (current subject line) | 18,500 | 1,480 | 8.00% |
| Treatment (new subject line) | 17,900 | 1,575 | 8.80% |
Use a two-sided test at significance level α = 0.05.
Requirements
- State the null and alternative hypotheses.
- Compute the sample proportions for control and treatment.
- Calculate the pooled proportion and the standard error for a two-proportion z-test.
- Compute the z-statistic and two-sided p-value.
- Determine whether the result is statistically significant at the 5% level.
- Calculate a 95% confidence interval for the difference in CTR.
- Briefly explain what the result means for the email rollout decision.
Assumptions
- Recipients were randomly assigned to control and treatment.
- Each recipient appears only once in the experiment.
- Click outcomes are independent within and across groups.
- Sample sizes are large enough for the normal approximation to be reasonable.
