Dataford
Popular roles
Software EngineerData AnalystData ScientistData EngineerBusiness AnalystAI EngineerMachine Learning EngineerProduct Manager
Browse
Browse All RolesEvery role hub, from analyst to MLBrowse All CompaniesCompany-specific interview loopsAll Interview GuidesThe full guide library
Top questions by role
Software EngineerData AnalystData ScientistData EngineerBusiness AnalystAI EngineerMachine Learning EngineerProduct Manager
Top questions by skill
SQLPythonStatisticsMachine LearningA/B TestingSystem DesignGenerative AIProduct SenseMetricsBehavioral
Browse all questions →Try a mock interview
ExperiencesCoaching
New
Practice
Mock InterviewsTimed interview simulations with feedbackSuccess PathYour 6-week structured planModulesCurated lessons by topicWebinarsTalks from ex-Big Tech data leadsPlaygroundA free-form scratch editor
Learn
BlogInterview strategy and career adviceTech Job Market ReportHiring trends across data and AI rolesFor UniversitiesDataford for career centersAbout DatafordWho we are and how we build
Pricing
Get started

MDE Planning for Splice Signup Test

HardStatistics & Probability00:00
Practice interviewer
In session
5 left
00:00

Whenever you're ready, walk me through how you'd approach this.

You need to log in / sign up to chat or submit.

Problem

Business Context

Splice is considering an experiment on the Sounds landing page signup flow to increase the rate of visitors who create an account and start a free trial. Before launching, the Growth team wants to choose a minimum detectable effect (MDE) that is realistic, statistically defensible, and worth the engineering cost.

Problem Statement

You are planning a two-arm A/B test with equal traffic split between the current signup flow (control) and a redesigned flow (treatment). Use the baseline conversion rate and traffic estimates below to determine the smallest lift the team can reliably detect in a 21-day experiment, and whether a proposed target lift is feasible.

Given Data

MetricValue
Baseline visitor-to-trial conversion rate8.4%
Average eligible unique visitors per day52,000
Planned experiment duration21 days
Traffic split50% control / 50% treatment
Significance level0.05
Desired power80%
Proposed business target lift0.8 percentage points
Minimum worthwhile lift from business perspective0.6 percentage points

Assume the primary metric is binary conversion at the user level.

Requirements

  1. Compute the per-variant sample size available in 21 days.
  2. Estimate the MDE in absolute percentage points for a two-sided test at the stated  and power.
  3. Convert that MDE to a relative lift over baseline.
  4. Determine whether the proposed 0.8 percentage point lift is detectable with the planned runtime.
  5. If the team wants to detect a 0.6 percentage point lift instead, estimate the required sample size per arm and approximate test duration.
  6. Briefly explain how you would think about choosing MDE in practice at Splice beyond just plugging numbers into a formula.

Assumptions

  • Random assignment is valid and independent across users.
  • Use the normal approximation for power/sample-size calculations for two proportions.
  • Ignore variance reduction methods such as CUPED.
  • No peeking or early stopping adjustments.