Email Click Rate Significance Test

Business Context

BrightCart, an online retail platform, wants to know whether a new promotional email template improves click-through rate versus the current template. The marketing team ran a randomized A/B test on a single campaign.

Problem Statement

Use an appropriate statistical method to determine whether the new email template produced a statistically significant change in click-through rate.

Given Data

Additional test settings:

Requirements

1. State the null and alternative hypotheses.
2. Identify the statistical method you would use and why it is appropriate.
3. Calculate the pooled click-through rate.
4. Compute the standard error and z-statistic.
5. Calculate the two-tailed p-value.
6. Construct a 95% confidence interval for the difference in click-through rates.
7. Conclude whether BrightCart should treat the result as statistically significant.

Assumptions

• Users were randomly assigned to control and treatment.
• Each email outcome is independent.
• Sample sizes are large enough for the normal approximation to hold.
• The metric of interest is binary at the user level: clicked or did not click.