P-Value vs Confidence Interval Communication | Dataford Interview Questions

Welcome to your interview for the Data Scientist role at Google.

The question is on your right: P-Value vs Confidence Interval Communication. Take a moment with it first.

Talk your thinking through with me if you like - when you're confident, submit your answer and I'll grade it like a real screen. You have three graded attempts to score 7/10 or better.

Business Context

StreamCart ran an A/B test on a new mobile checkout layout. The product manager wants a recommendation not only on whether the result is statistically significant, but also on whether to communicate the outcome using a p-value, a confidence interval, or both.

Problem Statement

Analyze the experiment result for conversion rate and decide how to communicate it to two audiences: (1) the product manager deciding on rollout and (2) the executive team reviewing experiment impact.

Given Data

Group	Sample Size	Conversions	Conversion Rate
Control	18,400	2,208	12.0%
Treatment	18,100	2,317	12.8%

Additional inputs:

Parameter	Value
Significance level	0.05
Confidence level	95%

The observed lift is 0.8 percentage points.

Requirements

State the null and alternative hypotheses for a two-sided test.
Compute the pooled proportion and the standard error for the hypothesis test.
Calculate the z-statistic and two-sided p-value.
Compute the 95% confidence interval for the difference in conversion rates.
Decide whether the result is statistically significant at the 5% level.
Explain when a p-value is the better communication tool and when a confidence interval is better.
Recommend what StreamCart should report to the product manager and to executives.

Assumptions

Users were randomly assigned to control and treatment.
Each observation is independent.
Sample sizes are large enough for normal approximation.
No major instrumentation or logging issues affected conversion counts.