Evaluate Marketing Campaign Impact with A/B Testing

Business Context

MarketMakers, a digital marketing agency, launched a new campaign aimed at improving the conversion rates of its client’s website. They conducted an A/B test over 30 days, with one group receiving the new marketing materials (treatment) and the other group receiving the old materials (control).

Given Data

Task

Assess whether the observed increase in conversion rate of 4.5 percentage points from the marketing campaign is statistically significant.

Requirements

1. State the null and alternative hypotheses.
2. Calculate the pooled conversion rate and standard error.
3. Compute the z-test statistic.
4. Find the p-value (two-tailed test).
5. Decide whether to reject H₀ at α = 0.05.
6. Provide a business interpretation of the results: should MarketMakers recommend the new marketing materials?

Assumptions

• Users were randomly assigned to control/treatment groups.
• Each user is counted only once (no repeat visits).
• The test duration was sufficient to account for daily fluctuations in traffic.