Interview Guides

Significance Test for Signup Conversion Change | Dataford Interview Questions - Dataford - Ace your Interview

Significance Test for Signup Conversion Change

Medium

Statistics & Probability

Confidence IntervalsP-ValuesStatistical Significance

Problem

Business Context

StreamCart changed its mobile signup landing page and wants to know whether the observed conversion lift is statistically meaningful or just random variation.

Problem Statement

You are given pre-change and post-change user-level conversion results from a 10-day measurement window. Treat this as a comparison of two independent proportions and determine whether the metric change is statistically significant.

Given Data

Group	Sample Size	Signups	Conversion Rate
Before change	18,420	2,026	11.00%
After change	18,105	2,118	11.70%

Additional parameters:

Parameter	Value
Significance level	0.05
Test type	Two-tailed
Confidence level	95%

Requirements

State the null and alternative hypotheses.
Compute the sample conversion rates and the observed absolute lift.
Calculate the pooled proportion and the standard error for a two-proportion z-test.
Compute the z-statistic and two-sided p-value.
Construct a 95% confidence interval for the difference in conversion rates.
Decide whether the change is statistically meaningful or likely noise.
Briefly explain whether the result is practically meaningful for the business.

Assumptions

Users in the before and after periods are independent observations.
Traffic mix is comparable across periods.
No major concurrent marketing or pricing changes affected signup behavior.
Sample sizes are large enough for normal approximation to be reasonable.

Sample Data

Example 1

Input

{"alpha":0.05,"after_n":18105,"before_n":18420,"after_rate":0.117,"before_rate":0.11,"after_signups":2118,"before_signups":2026,"confidence_level":0.95}

Output(none)

Problem

Business Context

StreamCart changed its mobile signup landing page and wants to know whether the observed conversion lift is statistically meaningful or just random variation.

Problem Statement

Given Data

Group	Sample Size	Signups	Conversion Rate
Before change	18,420	2,026	11.00%
After change	18,105	2,118	11.70%

Additional parameters:

Parameter	Value
Significance level	0.05
Test type	Two-tailed
Confidence level	95%

Requirements

State the null and alternative hypotheses.
Compute the sample conversion rates and the observed absolute lift.
Calculate the pooled proportion and the standard error for a two-proportion z-test.
Compute the z-statistic and two-sided p-value.
Construct a 95% confidence interval for the difference in conversion rates.
Decide whether the change is statistically meaningful or likely noise.
Briefly explain whether the result is practically meaningful for the business.

Assumptions

Users in the before and after periods are independent observations.
Traffic mix is comparable across periods.
No major concurrent marketing or pricing changes affected signup behavior.
Sample sizes are large enough for normal approximation to be reasonable.

Sample Data

Example 1

Input

{"alpha":0.05,"after_n":18105,"before_n":18420,"after_rate":0.117,"before_rate":0.11,"after_signups":2118,"before_signups":2026,"confidence_level":0.95}

Output(none)

Your answer

Try one AI text evaluation on us

Get structured feedback, scored against a 4-axis rubric. Premium unlocks unlimited.

0 wordstarget ~200

Up next

Significance Test for Signup LiftMedium ASignificance Test for Signup ConversionEasy

Conversion Significance for Signup RedesignEasy

Next question

Significance Test for Signup Conversion Change

Medium

Statistics & Probability

Confidence IntervalsP-ValuesStatistical Significance

Problem

Business Context

StreamCart changed its mobile signup landing page and wants to know whether the observed conversion lift is statistically meaningful or just random variation.

Problem Statement

Given Data

Group	Sample Size	Signups	Conversion Rate
Before change	18,420	2,026	11.00%
After change	18,105	2,118	11.70%

Additional parameters:

Parameter	Value
Significance level	0.05
Test type	Two-tailed
Confidence level	95%

Requirements

State the null and alternative hypotheses.
Compute the sample conversion rates and the observed absolute lift.
Calculate the pooled proportion and the standard error for a two-proportion z-test.
Compute the z-statistic and two-sided p-value.
Construct a 95% confidence interval for the difference in conversion rates.
Decide whether the change is statistically meaningful or likely noise.
Briefly explain whether the result is practically meaningful for the business.

Assumptions

Users in the before and after periods are independent observations.
Traffic mix is comparable across periods.
No major concurrent marketing or pricing changes affected signup behavior.
Sample sizes are large enough for normal approximation to be reasonable.

Sample Data

Example 1

Input

{"alpha":0.05,"after_n":18105,"before_n":18420,"after_rate":0.117,"before_rate":0.11,"after_signups":2118,"before_signups":2026,"confidence_level":0.95}

Output(none)

Problem

Business Context

StreamCart changed its mobile signup landing page and wants to know whether the observed conversion lift is statistically meaningful or just random variation.

Problem Statement

Given Data

Group	Sample Size	Signups	Conversion Rate
Before change	18,420	2,026	11.00%
After change	18,105	2,118	11.70%

Additional parameters:

Parameter	Value
Significance level	0.05
Test type	Two-tailed
Confidence level	95%

Requirements

State the null and alternative hypotheses.
Compute the sample conversion rates and the observed absolute lift.
Calculate the pooled proportion and the standard error for a two-proportion z-test.
Compute the z-statistic and two-sided p-value.
Construct a 95% confidence interval for the difference in conversion rates.
Decide whether the change is statistically meaningful or likely noise.
Briefly explain whether the result is practically meaningful for the business.

Assumptions

Users in the before and after periods are independent observations.
Traffic mix is comparable across periods.
No major concurrent marketing or pricing changes affected signup behavior.
Sample sizes are large enough for normal approximation to be reasonable.

Sample Data

Example 1

Input

{"alpha":0.05,"after_n":18105,"before_n":18420,"after_rate":0.117,"before_rate":0.11,"after_signups":2118,"before_signups":2026,"confidence_level":0.95}

Output(none)

Your answer

Try one AI text evaluation on us

Get structured feedback, scored against a 4-axis rubric. Premium unlocks unlimited.

0 wordstarget ~200

Up next

Significance Test for Signup LiftMedium ASignificance Test for Signup ConversionEasy

Conversion Significance for Signup RedesignEasy

Next question