Interview Guides

Evaluate Marketing Campaign Impact with A/B Testing | Dataford Interview Questions - Dataford - Ace your Interview

Evaluate Marketing Campaign Impact with A/B Testing

Hard

Statistics & Probability

Asked at 49 companies49Statistical SignificanceExperimentationCausal Inference

Also asked at

Problem

Business Context

MarketMakers, a digital marketing agency, launched a new campaign aimed at improving the conversion rates of its client’s website. They conducted an A/B test over 30 days, with one group receiving the new marketing materials (treatment) and the other group receiving the old materials (control).

Given Data

Group	Sample Size	Conversions	Conversion Rate
Control	50,000	6,250	12.5%
Treatment	15,000	2,550	17.0%

Task

Assess whether the observed increase in conversion rate of 4.5 percentage points from the marketing campaign is statistically significant.

Requirements

State the null and alternative hypotheses.
Calculate the pooled conversion rate and standard error.
Compute the z-test statistic.
Find the p-value (two-tailed test).
Decide whether to reject H₀ at α = 0.05.
Provide a business interpretation of the results: should MarketMakers recommend the new marketing materials?

Assumptions

Users were randomly assigned to control/treatment groups.
Each user is counted only once (no repeat visits).
The test duration was sufficient to account for daily fluctuations in traffic.

Sample Data

Example 1

Input

{"significance_level":0.05,"control_conversions":6250,"control_sample_size":50000,"treatment_conversions":2550,"treatment_sample_size":15000}

Output(none)

Problem

Business Context

Given Data

Group	Sample Size	Conversions	Conversion Rate
Control	50,000	6,250	12.5%
Treatment	15,000	2,550	17.0%

Task

Assess whether the observed increase in conversion rate of 4.5 percentage points from the marketing campaign is statistically significant.

Requirements

State the null and alternative hypotheses.
Calculate the pooled conversion rate and standard error.
Compute the z-test statistic.
Find the p-value (two-tailed test).
Decide whether to reject H₀ at α = 0.05.
Provide a business interpretation of the results: should MarketMakers recommend the new marketing materials?

Assumptions

Users were randomly assigned to control/treatment groups.
Each user is counted only once (no repeat visits).
The test duration was sufficient to account for daily fluctuations in traffic.

Sample Data

Example 1

Input

{"significance_level":0.05,"control_conversions":6250,"control_sample_size":50000,"treatment_conversions":2550,"treatment_sample_size":15000}

Output(none)

Your answer

Try one AI text evaluation on us

Get structured feedback, scored against a 4-axis rubric. Premium unlocks unlimited.

0 wordstarget ~200

Up next

Evaluate A/B Test Results for Email CampaignEasy

Marketing Email A/B Test SignificanceEasy A/B Test for Email Campaign LiftEasy

Next question

Evaluate Marketing Campaign Impact with A/B Testing

Hard

Statistics & Probability

Asked at 49 companies49Statistical SignificanceExperimentationCausal Inference

Also asked at

Problem

Business Context

Given Data

Group	Sample Size	Conversions	Conversion Rate
Control	50,000	6,250	12.5%
Treatment	15,000	2,550	17.0%

Task

Assess whether the observed increase in conversion rate of 4.5 percentage points from the marketing campaign is statistically significant.

Requirements

State the null and alternative hypotheses.
Calculate the pooled conversion rate and standard error.
Compute the z-test statistic.
Find the p-value (two-tailed test).
Decide whether to reject H₀ at α = 0.05.
Provide a business interpretation of the results: should MarketMakers recommend the new marketing materials?

Assumptions

Users were randomly assigned to control/treatment groups.
Each user is counted only once (no repeat visits).
The test duration was sufficient to account for daily fluctuations in traffic.

Sample Data

Example 1

Input

{"significance_level":0.05,"control_conversions":6250,"control_sample_size":50000,"treatment_conversions":2550,"treatment_sample_size":15000}

Output(none)

Problem

Business Context

Given Data

Group	Sample Size	Conversions	Conversion Rate
Control	50,000	6,250	12.5%
Treatment	15,000	2,550	17.0%

Task

Assess whether the observed increase in conversion rate of 4.5 percentage points from the marketing campaign is statistically significant.

Requirements

State the null and alternative hypotheses.
Calculate the pooled conversion rate and standard error.
Compute the z-test statistic.
Find the p-value (two-tailed test).
Decide whether to reject H₀ at α = 0.05.
Provide a business interpretation of the results: should MarketMakers recommend the new marketing materials?

Assumptions

Users were randomly assigned to control/treatment groups.
Each user is counted only once (no repeat visits).
The test duration was sufficient to account for daily fluctuations in traffic.

Sample Data

Example 1

Input

{"significance_level":0.05,"control_conversions":6250,"control_sample_size":50000,"treatment_conversions":2550,"treatment_sample_size":15000}

Output(none)

Your answer

Try one AI text evaluation on us

Get structured feedback, scored against a 4-axis rubric. Premium unlocks unlimited.

0 wordstarget ~200

Up next

Evaluate A/B Test Results for Email CampaignEasy

Marketing Email A/B Test SignificanceEasy A/B Test for Email Campaign LiftEasy

Next question