Interview Guides

A/B Test for Chime Sign-Up Flow | Dataford Interview Questions - Dataford - Ace your Interview

A/B Test for Chime Sign-Up Flow

Medium

Statistics & Probability

Asked at 1 company1Hypothesis TestingA/B TestingExperimentation

Also asked at

Problem

Business Context

Chime wants to test a redesigned sign-up flow on its landing page to increase completed account applications without hurting downstream quality. A 14-day randomized experiment was run on new visitors to Chime's landing page.

Problem Statement

Evaluate whether the new sign-up flow improves the primary conversion metric enough to justify rollout, and quantify the uncertainty around the lift.

Given Data

Group	Visitors	Completed Sign-Ups	Conversion Rate
Control (current flow)	52,400	6,131	11.7004%
Treatment (new flow)	52,050	6,534	12.5533%

Additional experiment parameters:

Parameter	Value
Significance level	0.05
Test type	Two-sided
Planned power	0.80
Baseline conversion for planning	11.7%
Minimum detectable effect for planning	0.8 percentage points

Requirements

State the null and alternative hypotheses for the sign-up conversion rate.
Compute the observed conversion rates and absolute lift.
Run a two-proportion z-test using the pooled standard error.
Calculate the two-sided p-value.
Construct a 95% confidence interval for the difference in conversion rates.
Determine whether the result is statistically significant at the 5% level.
Briefly assess whether the observed lift is practically meaningful for Chime.
Using the planning assumptions, estimate the required sample size per variant for 80% power.

Assumptions

Visitors were randomly assigned at the user level and counted once.
No major instrumentation issues or sample-ratio mismatch occurred.
The normal approximation is valid because both groups have large sample sizes.
The primary metric is completed sign-up conversion on the landing-page flow.

Sample Data

Example 1

Input

{"alpha":0.05,"power":0.8,"control_n":52400,"treatment_n":52050,"control_rate":0.117004,"mde_absolute":0.008,"baseline_rate":0.117,"treatment_rate":0.125533,"control_conversions":6131,"treatment_conversions":6534}

Output(none)

Problem

Business Context

Problem Statement

Evaluate whether the new sign-up flow improves the primary conversion metric enough to justify rollout, and quantify the uncertainty around the lift.

Given Data

Group	Visitors	Completed Sign-Ups	Conversion Rate
Control (current flow)	52,400	6,131	11.7004%
Treatment (new flow)	52,050	6,534	12.5533%

Additional experiment parameters:

Parameter	Value
Significance level	0.05
Test type	Two-sided
Planned power	0.80
Baseline conversion for planning	11.7%
Minimum detectable effect for planning	0.8 percentage points

Requirements

State the null and alternative hypotheses for the sign-up conversion rate.
Compute the observed conversion rates and absolute lift.
Run a two-proportion z-test using the pooled standard error.
Calculate the two-sided p-value.
Construct a 95% confidence interval for the difference in conversion rates.
Determine whether the result is statistically significant at the 5% level.
Briefly assess whether the observed lift is practically meaningful for Chime.
Using the planning assumptions, estimate the required sample size per variant for 80% power.

Assumptions

Visitors were randomly assigned at the user level and counted once.
No major instrumentation issues or sample-ratio mismatch occurred.
The normal approximation is valid because both groups have large sample sizes.
The primary metric is completed sign-up conversion on the landing-page flow.

Sample Data

Example 1

Input

{"alpha":0.05,"power":0.8,"control_n":52400,"treatment_n":52050,"control_rate":0.117004,"mde_absolute":0.008,"baseline_rate":0.117,"treatment_rate":0.125533,"control_conversions":6131,"treatment_conversions":6534}

Output(none)

Your answer

Try one AI text evaluation on us

Get structured feedback, scored against a 4-axis rubric. Premium unlocks unlimited.

0 wordstarget ~200

Up next

A/B Test Validation for Signup RedesignEasy

Launch Decision for Signup ExperimentMedium Conversion Lift Significance for Signup RedesignMedium

Next question

A/B Test for Chime Sign-Up Flow

Medium

Statistics & Probability

Asked at 1 company1Hypothesis TestingA/B TestingExperimentation

Also asked at

Problem

Business Context

Problem Statement

Evaluate whether the new sign-up flow improves the primary conversion metric enough to justify rollout, and quantify the uncertainty around the lift.

Given Data

Group	Visitors	Completed Sign-Ups	Conversion Rate
Control (current flow)	52,400	6,131	11.7004%
Treatment (new flow)	52,050	6,534	12.5533%

Additional experiment parameters:

Parameter	Value
Significance level	0.05
Test type	Two-sided
Planned power	0.80
Baseline conversion for planning	11.7%
Minimum detectable effect for planning	0.8 percentage points

Requirements

State the null and alternative hypotheses for the sign-up conversion rate.
Compute the observed conversion rates and absolute lift.
Run a two-proportion z-test using the pooled standard error.
Calculate the two-sided p-value.
Construct a 95% confidence interval for the difference in conversion rates.
Determine whether the result is statistically significant at the 5% level.
Briefly assess whether the observed lift is practically meaningful for Chime.
Using the planning assumptions, estimate the required sample size per variant for 80% power.

Assumptions

Visitors were randomly assigned at the user level and counted once.
No major instrumentation issues or sample-ratio mismatch occurred.
The normal approximation is valid because both groups have large sample sizes.
The primary metric is completed sign-up conversion on the landing-page flow.

Sample Data

Example 1

Input

{"alpha":0.05,"power":0.8,"control_n":52400,"treatment_n":52050,"control_rate":0.117004,"mde_absolute":0.008,"baseline_rate":0.117,"treatment_rate":0.125533,"control_conversions":6131,"treatment_conversions":6534}

Output(none)

Problem

Business Context

Problem Statement

Evaluate whether the new sign-up flow improves the primary conversion metric enough to justify rollout, and quantify the uncertainty around the lift.

Given Data

Group	Visitors	Completed Sign-Ups	Conversion Rate
Control (current flow)	52,400	6,131	11.7004%
Treatment (new flow)	52,050	6,534	12.5533%

Additional experiment parameters:

Parameter	Value
Significance level	0.05
Test type	Two-sided
Planned power	0.80
Baseline conversion for planning	11.7%
Minimum detectable effect for planning	0.8 percentage points

Requirements

State the null and alternative hypotheses for the sign-up conversion rate.
Compute the observed conversion rates and absolute lift.
Run a two-proportion z-test using the pooled standard error.
Calculate the two-sided p-value.
Construct a 95% confidence interval for the difference in conversion rates.
Determine whether the result is statistically significant at the 5% level.
Briefly assess whether the observed lift is practically meaningful for Chime.
Using the planning assumptions, estimate the required sample size per variant for 80% power.

Assumptions

Visitors were randomly assigned at the user level and counted once.
No major instrumentation issues or sample-ratio mismatch occurred.
The normal approximation is valid because both groups have large sample sizes.
The primary metric is completed sign-up conversion on the landing-page flow.

Sample Data

Example 1

Input

{"alpha":0.05,"power":0.8,"control_n":52400,"treatment_n":52050,"control_rate":0.117004,"mde_absolute":0.008,"baseline_rate":0.117,"treatment_rate":0.125533,"control_conversions":6131,"treatment_conversions":6534}

Output(none)

Your answer

Try one AI text evaluation on us

Get structured feedback, scored against a 4-axis rubric. Premium unlocks unlimited.

0 wordstarget ~200

Up next

A/B Test Validation for Signup RedesignEasy

Launch Decision for Signup ExperimentMedium Conversion Lift Significance for Signup RedesignMedium

Next question