Dataford
Popular roles
Software EngineerData AnalystData ScientistData EngineerBusiness AnalystAI EngineerMachine Learning EngineerProduct Manager
Browse
Browse All RolesEvery role hub, from analyst to MLBrowse All CompaniesCompany-specific interview loopsAll Interview GuidesThe full guide library
Top questions by role
Software EngineerData AnalystData ScientistData EngineerBusiness AnalystAI EngineerMachine Learning EngineerProduct Manager
Top questions by skill
SQLPythonStatisticsMachine LearningA/B TestingSystem DesignGenerative AIProduct SenseMetricsBehavioral
Browse all questions →Try a mock interview
ExperiencesCoaching
New
Practice
Mock InterviewsTimed interview simulations with feedbackSuccess PathYour 6-week structured planModulesCurated lessons by topicWebinarsTalks from ex-Big Tech data leadsPlaygroundA free-form scratch editor
Learn
BlogInterview strategy and career adviceTech Job Market ReportHiring trends across data and AI rolesFor UniversitiesDataford for career centersAbout DatafordWho we are and how we build
Pricing
Get started

Experiment Duration for Signup Lift

MediumStatistics & Probability00:00
Practice interviewer
In session
5 left
00:00

Whenever you're ready, walk me through how you'd approach this.

You need to log in / sign up to chat or submit.

Problem

Business Context

StreamCart is testing a new signup page and wants to know how long the A/B test must run before it can reliably detect the expected improvement. The primary metric is visitor-to-signup conversion.

Problem Statement

The team expects a 2% relative lift over a 10% baseline conversion rate, so the treatment conversion rate is expected to be 10.2%. Estimate the required sample size per variant and convert that into experiment duration.

Given Data

MetricValue
Baseline conversion rate10.0%
Expected relative lift2.0%
Expected treatment conversion rate10.2%
Absolute lift0.2 percentage points
Significance level5.0%
Statistical power80.0%
Allocation50% control / 50% treatment
Daily eligible visitors500,000
Share included in experiment100%

Requirements

  1. Convert the relative lift into an absolute effect size.
  2. Compute the required sample size per group for a two-sample test of proportions.
  3. Estimate the total sample size needed across both groups.
  4. Convert the sample requirement into the number of days the experiment should run.
  5. Briefly explain whether the answer changes if the team wants to capture full weekly seasonality.

Assumptions

  • Random assignment at the user level with no interference between users.
  • Two-sided hypothesis test with constant traffic over time.
  • Normal approximation for two proportions is appropriate at this scale.
  • Ignore attrition, sample ratio mismatch, and metric definition changes during the test.