Business Context

PulseMail, a marketing automation platform, wants to know whether a new subject line meaningfully improves email click-through rate (CTR). The analytics team ran a randomized A/B test on a campaign sent to similar customer segments.

Problem Statement

Use an appropriate statistical method to determine whether the new subject line produced a statistically significant change in CTR versus the original subject line.

Given Data

Additional test settings:

Requirements

1. State the null and alternative hypotheses.
2. Identify the statistical method you would use and why it is appropriate.
3. Calculate the sample proportions for both groups.
4. Compute the pooled proportion and standard error.
5. Calculate the test statistic and two-tailed p-value.
6. Construct a 95% confidence interval for the difference in CTR.
7. Conclude whether PulseMail should treat the result as statistically significant.

Assumptions

• Users were randomly assigned to control and treatment.
• Each delivered email corresponds to one independent observation.
• The normal approximation is valid because both groups have sufficiently large sample sizes and enough clicks/non-clicks.
• No major deliverability issues or targeting changes occurred during the experiment.