Business Context

BrightCart, an online retailer, tested a new promotional email subject line to improve purchase conversion from a marketing campaign. Users were randomly assigned to receive either the current subject line (control) or the new subject line (treatment).

Problem Statement

Determine whether the new subject line produced a statistically significant improvement in conversion rate, and quantify the size of the lift.

Given Data

Use a two-sided test with significance level $\alpha = 0.05$.

Requirements

1. State the null and alternative hypotheses.
2. Compute the sample conversion rates and the observed lift.
3. Calculate the pooled proportion and standard error for a two-proportion z-test.
4. Compute the z-statistic and two-sided p-value.
5. Construct a 95% confidence interval for the difference in conversion rates.
6. Decide whether BrightCart should roll out the new subject line to all campaign traffic.

Assumptions

• Random assignment was implemented correctly.
• Each user appears once and outcomes are independent across users.
• The normal approximation is appropriate because both groups have sufficiently large sample sizes and enough conversions/non-conversions.
• No major campaign execution issues affected one group differently from the other.