Interview Guides

Campaign Lift Significance for Email Offer | Dataford Interview Questions - Dataford - Ace your Interview

Campaign Lift Significance for Email Offer

Easy

Statistics & Probability

Asked at 3 companies3Hypothesis TestingP-ValuesStatistical Significance

Also asked at

Problem

Business Context

BrightCart ran an email campaign offering free shipping to a randomly selected group of users. The marketing team wants to know whether the observed lift in purchase conversion is real or just sampling noise.

Problem Statement

Use the campaign test results to determine whether the campaign's lift in conversion rate is statistically significant.

Given Data

Group	Sample Size	Purchases	Conversion Rate
Control (no email)	18,500	1,554	8.40%
Treatment (email sent)	17,900	1,665	9.30%

Additional parameters:

Parameter	Value
Significance level	0.05
Test type	Two-tailed

The observed lift is 0.93% - 0.84% = 0.009, or 0.9 percentage points.

Requirements

State the null and alternative hypotheses for the campaign lift.
Compute the sample conversion rates and the pooled proportion.
Calculate the standard error for a two-proportion z-test.
Compute the z-statistic and p-value.
Construct a 95% confidence interval for the lift in conversion rate.
Decide whether the lift is statistically significant at the 5% level.
Briefly explain whether the result is also practically meaningful for the business.

Assumptions

Users were randomly assigned to treatment and control.
Each user contributes at most one purchase outcome.
The normal approximation is valid because both groups have large sample sizes and enough conversions/non-conversions.
No major targeting or delivery bias affected which users received the campaign.

Sample Data

Example 1

Input

{"observed_lift":0.009,"significance_level":0.05,"control_conversions":1554,"control_sample_size":18500,"treatment_conversions":1665,"treatment_sample_size":17900,"control_conversion_rate":0.084,"treatment_conversion_rate":0.093}

Output(none)

Problem

Business Context

Problem Statement

Use the campaign test results to determine whether the campaign's lift in conversion rate is statistically significant.

Given Data

Group	Sample Size	Purchases	Conversion Rate
Control (no email)	18,500	1,554	8.40%
Treatment (email sent)	17,900	1,665	9.30%

Additional parameters:

Parameter	Value
Significance level	0.05
Test type	Two-tailed

The observed lift is 0.93% - 0.84% = 0.009, or 0.9 percentage points.

Requirements

State the null and alternative hypotheses for the campaign lift.
Compute the sample conversion rates and the pooled proportion.
Calculate the standard error for a two-proportion z-test.
Compute the z-statistic and p-value.
Construct a 95% confidence interval for the lift in conversion rate.
Decide whether the lift is statistically significant at the 5% level.
Briefly explain whether the result is also practically meaningful for the business.

Assumptions

Users were randomly assigned to treatment and control.
Each user contributes at most one purchase outcome.
The normal approximation is valid because both groups have large sample sizes and enough conversions/non-conversions.
No major targeting or delivery bias affected which users received the campaign.

Sample Data

Example 1

Input

{"observed_lift":0.009,"significance_level":0.05,"control_conversions":1554,"control_sample_size":18500,"treatment_conversions":1665,"treatment_sample_size":17900,"control_conversion_rate":0.084,"treatment_conversion_rate":0.093}

Output(none)

Your answer

Try one AI text evaluation on us

Get structured feedback, scored against a 4-axis rubric. Premium unlocks unlimited.

0 wordstarget ~200

Up next

AA/B Test for Email Campaign LiftEasy A/B Test for Email Campaign LiftEasy

Marketing Email A/B Test SignificanceEasy

Next question

Campaign Lift Significance for Email Offer

Easy

Statistics & Probability

Asked at 3 companies3Hypothesis TestingP-ValuesStatistical Significance

Also asked at

Problem

Business Context

Problem Statement

Use the campaign test results to determine whether the campaign's lift in conversion rate is statistically significant.

Given Data

Group	Sample Size	Purchases	Conversion Rate
Control (no email)	18,500	1,554	8.40%
Treatment (email sent)	17,900	1,665	9.30%

Additional parameters:

Parameter	Value
Significance level	0.05
Test type	Two-tailed

The observed lift is 0.93% - 0.84% = 0.009, or 0.9 percentage points.

Requirements

State the null and alternative hypotheses for the campaign lift.
Compute the sample conversion rates and the pooled proportion.
Calculate the standard error for a two-proportion z-test.
Compute the z-statistic and p-value.
Construct a 95% confidence interval for the lift in conversion rate.
Decide whether the lift is statistically significant at the 5% level.
Briefly explain whether the result is also practically meaningful for the business.

Assumptions

Users were randomly assigned to treatment and control.
Each user contributes at most one purchase outcome.
The normal approximation is valid because both groups have large sample sizes and enough conversions/non-conversions.
No major targeting or delivery bias affected which users received the campaign.

Sample Data

Example 1

Input

{"observed_lift":0.009,"significance_level":0.05,"control_conversions":1554,"control_sample_size":18500,"treatment_conversions":1665,"treatment_sample_size":17900,"control_conversion_rate":0.084,"treatment_conversion_rate":0.093}

Output(none)

Problem

Business Context

Problem Statement

Use the campaign test results to determine whether the campaign's lift in conversion rate is statistically significant.

Given Data

Group	Sample Size	Purchases	Conversion Rate
Control (no email)	18,500	1,554	8.40%
Treatment (email sent)	17,900	1,665	9.30%

Additional parameters:

Parameter	Value
Significance level	0.05
Test type	Two-tailed

The observed lift is 0.93% - 0.84% = 0.009, or 0.9 percentage points.

Requirements

State the null and alternative hypotheses for the campaign lift.
Compute the sample conversion rates and the pooled proportion.
Calculate the standard error for a two-proportion z-test.
Compute the z-statistic and p-value.
Construct a 95% confidence interval for the lift in conversion rate.
Decide whether the lift is statistically significant at the 5% level.
Briefly explain whether the result is also practically meaningful for the business.

Assumptions

Users were randomly assigned to treatment and control.
Each user contributes at most one purchase outcome.
The normal approximation is valid because both groups have large sample sizes and enough conversions/non-conversions.
No major targeting or delivery bias affected which users received the campaign.

Sample Data

Example 1

Input

{"observed_lift":0.009,"significance_level":0.05,"control_conversions":1554,"control_sample_size":18500,"treatment_conversions":1665,"treatment_sample_size":17900,"control_conversion_rate":0.084,"treatment_conversion_rate":0.093}

Output(none)

Your answer

Try one AI text evaluation on us

Get structured feedback, scored against a 4-axis rubric. Premium unlocks unlimited.

0 wordstarget ~200

Up next

AA/B Test for Email Campaign LiftEasy A/B Test for Email Campaign LiftEasy

Marketing Email A/B Test SignificanceEasy

Next question