Interview Guides

Significance Test for Email CTR | Dataford Interview Questions - Dataford - Ace your Interview

Significance Test for Email CTR

Easy

Statistics & Probability

Hypothesis TestingP-ValuesStatistical Significance

Problem

Business Context

BrightMail, a marketing automation company, tested a new subject line to improve email click-through rate (CTR). The analytics team wants to know whether the observed lift is statistically significant or could be due to random variation.

Problem Statement

You are given results from a randomized A/B test comparing the current subject line (control) with a new subject line (treatment). Determine whether the treatment's higher CTR is statistically significant at the 5% level.

Given Data

Group	Emails Delivered	Clicks	Click-Through Rate
Control	18,500	1,554	8.40%
Treatment	17,900	1,665	9.30%

Use a two-proportion z-test with a two-sided alternative.

Requirements

State the null and alternative hypotheses.
Calculate the sample click-through rates for both groups.
Compute the pooled proportion under the null hypothesis.
Calculate the standard error for the difference in proportions.
Compute the z-statistic and two-sided p-value.
Decide whether the result is statistically significant at $\alpha = 0.05$ .
Briefly explain what the result means for the product marketing team.

Assumptions

Users were randomly assigned to control and treatment.
Each delivered email is an independent trial.
The normal approximation is appropriate because both groups have large sample sizes and sufficient numbers of clicks and non-clicks.
No major deliverability issue affected one group differently from the other.

Sample Data

Example 1

Input

{"alpha":0.05,"control_n":18500,"treatment_n":17900,"control_rate":0.084,"control_clicks":1554,"treatment_rate":0.093,"treatment_clicks":1665}

Output(none)

Problem

Business Context

Problem Statement

Given Data

Group	Emails Delivered	Clicks	Click-Through Rate
Control	18,500	1,554	8.40%
Treatment	17,900	1,665	9.30%

Use a two-proportion z-test with a two-sided alternative.

Requirements

State the null and alternative hypotheses.
Calculate the sample click-through rates for both groups.
Compute the pooled proportion under the null hypothesis.
Calculate the standard error for the difference in proportions.
Compute the z-statistic and two-sided p-value.
Decide whether the result is statistically significant at $\alpha = 0.05$ .
Briefly explain what the result means for the product marketing team.

Assumptions

Users were randomly assigned to control and treatment.
Each delivered email is an independent trial.
The normal approximation is appropriate because both groups have large sample sizes and sufficient numbers of clicks and non-clicks.
No major deliverability issue affected one group differently from the other.

Sample Data

Example 1

Input

{"alpha":0.05,"control_n":18500,"treatment_n":17900,"control_rate":0.084,"control_clicks":1554,"treatment_rate":0.093,"treatment_clicks":1665}

Output(none)

Your answer

Try one AI text evaluation on us

Get structured feedback, scored against a 4-axis rubric. Premium unlocks unlimited.

0 wordstarget ~200

Up next

Statistical Significance for Email CTR LiftEasy Significance Test for Email CTREasy Statistical Significance for Email CTREasy

Next question

Significance Test for Email CTR

Easy

Statistics & Probability

Hypothesis TestingP-ValuesStatistical Significance

Problem

Business Context

Problem Statement

Given Data

Group	Emails Delivered	Clicks	Click-Through Rate
Control	18,500	1,554	8.40%
Treatment	17,900	1,665	9.30%

Use a two-proportion z-test with a two-sided alternative.

Requirements

State the null and alternative hypotheses.
Calculate the sample click-through rates for both groups.
Compute the pooled proportion under the null hypothesis.
Calculate the standard error for the difference in proportions.
Compute the z-statistic and two-sided p-value.
Decide whether the result is statistically significant at $\alpha = 0.05$ .
Briefly explain what the result means for the product marketing team.

Assumptions

Users were randomly assigned to control and treatment.
Each delivered email is an independent trial.
The normal approximation is appropriate because both groups have large sample sizes and sufficient numbers of clicks and non-clicks.
No major deliverability issue affected one group differently from the other.

Sample Data

Example 1

Input

{"alpha":0.05,"control_n":18500,"treatment_n":17900,"control_rate":0.084,"control_clicks":1554,"treatment_rate":0.093,"treatment_clicks":1665}

Output(none)

Problem

Business Context

Problem Statement

Given Data

Group	Emails Delivered	Clicks	Click-Through Rate
Control	18,500	1,554	8.40%
Treatment	17,900	1,665	9.30%

Use a two-proportion z-test with a two-sided alternative.

Requirements

State the null and alternative hypotheses.
Calculate the sample click-through rates for both groups.
Compute the pooled proportion under the null hypothesis.
Calculate the standard error for the difference in proportions.
Compute the z-statistic and two-sided p-value.
Decide whether the result is statistically significant at $\alpha = 0.05$ .
Briefly explain what the result means for the product marketing team.

Assumptions

Users were randomly assigned to control and treatment.
Each delivered email is an independent trial.
The normal approximation is appropriate because both groups have large sample sizes and sufficient numbers of clicks and non-clicks.
No major deliverability issue affected one group differently from the other.

Sample Data

Example 1

Input

{"alpha":0.05,"control_n":18500,"treatment_n":17900,"control_rate":0.084,"control_clicks":1554,"treatment_rate":0.093,"treatment_clicks":1665}

Output(none)

Your answer

Try one AI text evaluation on us

Get structured feedback, scored against a 4-axis rubric. Premium unlocks unlimited.

0 wordstarget ~200

Up next

Statistical Significance for Email CTR LiftEasy Significance Test for Email CTREasy Statistical Significance for Email CTREasy

Next question