Business Context
BrightCart, an online retail platform, wants to know whether a new promotional email template improves click-through rate versus the current template. The marketing team ran a randomized A/B test on a single campaign.
Problem Statement
Use an appropriate statistical method to determine whether the new email template produced a statistically significant change in click-through rate.
Given Data
| Group | Emails Sent | Clicks | Click-Through Rate |
|---|
| Control (current template) | 18,500 | 1,554 | 8.40% |
| Treatment (new template) | 17,900 | 1,647 | 9.20% |
Additional test settings:
| Parameter | Value |
|---|
| Significance level | 0.05 |
| Test type | Two-tailed |
| |
Requirements
- State the null and alternative hypotheses.
- Identify the statistical method you would use and why it is appropriate.
- Calculate the pooled click-through rate.
- Compute the standard error and z-statistic.
- Calculate the two-tailed p-value.
- Construct a 95% confidence interval for the difference in click-through rates.
- Conclude whether BrightCart should treat the result as statistically significant.
Assumptions
- Users were randomly assigned to control and treatment.
- Each email outcome is independent.
- Sample sizes are large enough for the normal approximation to hold.
- The metric of interest is binary at the user level: clicked or did not click.
