MarketHub, an online marketplace, tested a new promotional email subject line to increase purchase conversion from an email campaign. The marketing lead wants to know whether the observed lift is real enough to influence rollout, and how the p-value should be interpreted in a business decision.
You are given the results of a randomized A/B test comparing the current subject line (control) with a new subject line (treatment). Determine whether the conversion lift is statistically significant and explain what the p-value does and does not mean for the business.
| Group | Sample Size | Purchases | Conversion Rate |
|---|---|---|---|
| Control | 8,200 | 656 | 8.00% |
| Treatment | 8,100 | 713 | 8.80% |
Use a two-proportion z-test with significance level α = 0.05.