Business Context
BrightAds, a digital advertising platform, launched a new email subject line and wants to know whether it improved click-through rate versus the old version. The analytics team needs a statistically sound recommendation before rolling it out to all users.
Problem Statement
Use a two-proportion hypothesis test to determine whether the new subject line changed click-through rate. Also quantify the size of the effect with a 95% confidence interval.
Given Data
| Group | Emails Sent | Clicks | Click-Through Rate |
|---|
| Control (old subject line) | 18,450 | 1,328 | 7.20% |
| Treatment (new subject line) | 17,980 | 1,404 | 7.81% |
Use a two-tailed test with significance level α=0.05.
Requirements
- State the null and alternative hypotheses.
- Compute the sample click rates for both groups.
- Calculate the pooled proportion and pooled standard error for a two-proportion z-test.
- Compute the z-statistic and two-tailed p-value.
- Construct a 95% confidence interval for the difference in click-through rates.
- Decide whether the result is statistically significant at the 5% level.
- Explain whether the observed lift is meaningful enough to recommend rollout.
Assumptions
- Users were randomly assigned to control and treatment.
- Each email recipient is counted once.
- Click events are independent across users.
- Normal approximation is appropriate because both groups have large sample sizes and sufficient successes/failures.
