Explain P-Value in Marketing Test | Dataford Interview Questions

Welcome to your interview for the Marketing Analytics Specialist role at payactiv.

The question is on your right: Explain P-Value in Marketing Test. Take a moment with it first.

Talk your thinking through with me if you like - when you're confident, submit your answer and I'll grade it like a real screen. You have three graded attempts to score 7/10 or better.

Business Context

BrightCart, a retail app, tested a new promotional banner to increase email sign-ups. The VP of Marketing saw a p-value in the experiment readout and asked, "What does this actually mean in plain English?"

Problem Statement

You need to evaluate the A/B test statistically and then explain the p-value to a non-technical stakeholder without using misleading language. Use the data below to determine whether the banner likely changed sign-up behavior.

Given Data

Group	Sample Size	Sign-ups	Conversion Rate
Control (old banner)	8,400	1,008	12.0%
Treatment (new banner)	8,100	1,053	13.0%

Additional test settings:

Parameter	Value
Significance level	0.05
Test type	Two-tailed

Requirements

State the null and alternative hypotheses.
Compute the sample proportions for control and treatment.
Calculate the pooled proportion and standard error for a two-proportion z-test.
Compute the z-statistic and two-tailed p-value.
Decide whether the result is statistically significant at $\alpha = 0.05$ .
Write a 2-3 sentence explanation of the p-value for the VP of Marketing in plain English.
Briefly explain one common incorrect interpretation of a p-value.

Assumptions

Users were randomly assigned to control and treatment.
Each observation is independent.
Sample sizes are large enough for the normal approximation to hold.
No major instrumentation or tracking issues affected sign-up counts.