Interview Guides

Experiment Design for Lyft Ads Launch | Dataford Interview Questions - Dataford - Ace your Interview

Experiment Design for Lyft Ads Launch

Hard

Statistics & Probability

Asked at 1 company1A/B TestingExperimentationCausal Inference

Also asked at

Problem

Business Context

Lyft is testing a new in-app ad product in the Lyft rider app, shown on the post-ride receipt screen. The team wants to know whether the ad unit increases monetization enough to justify any degradation in rider experience.

Problem Statement

Design and analyze an experiment to estimate the causal impact of the new ad product on ad revenue per exposed rider while protecting user experience. Assume Lyft randomized eligible riders 50/50 into control (no ad on receipt screen) and treatment (new ad shown).

Given Data

Metric	Control	Treatment
Eligible riders	120,000	120,000
Riders with at least 1 completed ride	96,400	96,100
Total 7-day ad revenue	\$0	\$33,154.50
Mean 7-day total revenue per eligible rider (rides + ads)	\$18.420	\$18.696
Std. dev. of 7-day total revenue per eligible rider	\$12.10	\$12.40
Riders opening the app again within 7 days	42,388	40,842

Additional assumptions for planning:

Baseline 7-day re-open rate: 44.0%
Minimum detectable decline in re-open rate worth catching: 1.5 percentage points
Significance level: 0.05
Desired power: 80%

Requirements

Define a primary metric and at least one guardrail metric.
State null and alternative hypotheses for both monetization and user experience.
Test whether treatment increased 7-day total revenue per eligible rider.
Test whether treatment reduced 7-day re-open rate.
Compute a 95% confidence interval for the revenue lift and for the re-open-rate difference.
Estimate the required sample size per group to detect a 1.5 percentage point drop in re-open rate.
Give a launch recommendation and note key experimental risks.

Assumptions

Randomization is at the eligible rider level and stable over the 7-day window.
Revenue per rider is approximately normal by the central limit theorem at this sample size.
Re-open behavior is binary and independent across riders.

Sample Data

Example 1

Input

{"alpha":0.05,"power":0.8,"control_n":120000,"treatment_n":120000,"control_reopens":42388,"mde_reopen_drop":0.015,"treatment_reopens":40842,"control_sd_revenue":12.1,"baseline_reopen_rate":0.44,"control_mean_revenue":18.42,"treatment_sd_revenue":12.4,"treatment_mean_revenue":18.696,"treatment_ad_revenue_total":33154.5}

Output(none)

Problem

Business Context

Problem Statement

Given Data

Metric	Control	Treatment
Eligible riders	120,000	120,000
Riders with at least 1 completed ride	96,400	96,100
Total 7-day ad revenue	\$0	\$33,154.50
Mean 7-day total revenue per eligible rider (rides + ads)	\$18.420	\$18.696
Std. dev. of 7-day total revenue per eligible rider	\$12.10	\$12.40
Riders opening the app again within 7 days	42,388	40,842

Additional assumptions for planning:

Baseline 7-day re-open rate: 44.0%
Minimum detectable decline in re-open rate worth catching: 1.5 percentage points
Significance level: 0.05
Desired power: 80%

Requirements

Define a primary metric and at least one guardrail metric.
State null and alternative hypotheses for both monetization and user experience.
Test whether treatment increased 7-day total revenue per eligible rider.
Test whether treatment reduced 7-day re-open rate.
Compute a 95% confidence interval for the revenue lift and for the re-open-rate difference.
Estimate the required sample size per group to detect a 1.5 percentage point drop in re-open rate.
Give a launch recommendation and note key experimental risks.

Assumptions

Randomization is at the eligible rider level and stable over the 7-day window.
Revenue per rider is approximately normal by the central limit theorem at this sample size.
Re-open behavior is binary and independent across riders.

Sample Data

Example 1

Input

{"alpha":0.05,"power":0.8,"control_n":120000,"treatment_n":120000,"control_reopens":42388,"mde_reopen_drop":0.015,"treatment_reopens":40842,"control_sd_revenue":12.1,"baseline_reopen_rate":0.44,"control_mean_revenue":18.42,"treatment_sd_revenue":12.4,"treatment_mean_revenue":18.696,"treatment_ad_revenue_total":33154.5}

Output(none)

Your answer

Try one AI text evaluation on us

Get structured feedback, scored against a 4-axis rubric. Premium unlocks unlimited.

0 wordstarget ~200

Up next

LA/B Test for Lyft Ride Request ConversionMedium LTest In-App Ad PlacementMedium LPower Lyft Rider Promo TestHard

Next question

Experiment Design for Lyft Ads Launch

Hard

Statistics & Probability

Asked at 1 company1A/B TestingExperimentationCausal Inference

Also asked at

Problem

Business Context

Problem Statement

Given Data

Metric	Control	Treatment
Eligible riders	120,000	120,000
Riders with at least 1 completed ride	96,400	96,100
Total 7-day ad revenue	\$0	\$33,154.50
Mean 7-day total revenue per eligible rider (rides + ads)	\$18.420	\$18.696
Std. dev. of 7-day total revenue per eligible rider	\$12.10	\$12.40
Riders opening the app again within 7 days	42,388	40,842

Additional assumptions for planning:

Baseline 7-day re-open rate: 44.0%
Minimum detectable decline in re-open rate worth catching: 1.5 percentage points
Significance level: 0.05
Desired power: 80%

Requirements

Define a primary metric and at least one guardrail metric.
State null and alternative hypotheses for both monetization and user experience.
Test whether treatment increased 7-day total revenue per eligible rider.
Test whether treatment reduced 7-day re-open rate.
Compute a 95% confidence interval for the revenue lift and for the re-open-rate difference.
Estimate the required sample size per group to detect a 1.5 percentage point drop in re-open rate.
Give a launch recommendation and note key experimental risks.

Assumptions

Randomization is at the eligible rider level and stable over the 7-day window.
Revenue per rider is approximately normal by the central limit theorem at this sample size.
Re-open behavior is binary and independent across riders.

Sample Data

Example 1

Input

{"alpha":0.05,"power":0.8,"control_n":120000,"treatment_n":120000,"control_reopens":42388,"mde_reopen_drop":0.015,"treatment_reopens":40842,"control_sd_revenue":12.1,"baseline_reopen_rate":0.44,"control_mean_revenue":18.42,"treatment_sd_revenue":12.4,"treatment_mean_revenue":18.696,"treatment_ad_revenue_total":33154.5}

Output(none)

Problem

Business Context

Problem Statement

Given Data

Metric	Control	Treatment
Eligible riders	120,000	120,000
Riders with at least 1 completed ride	96,400	96,100
Total 7-day ad revenue	\$0	\$33,154.50
Mean 7-day total revenue per eligible rider (rides + ads)	\$18.420	\$18.696
Std. dev. of 7-day total revenue per eligible rider	\$12.10	\$12.40
Riders opening the app again within 7 days	42,388	40,842

Additional assumptions for planning:

Baseline 7-day re-open rate: 44.0%
Minimum detectable decline in re-open rate worth catching: 1.5 percentage points
Significance level: 0.05
Desired power: 80%

Requirements

Define a primary metric and at least one guardrail metric.
State null and alternative hypotheses for both monetization and user experience.
Test whether treatment increased 7-day total revenue per eligible rider.
Test whether treatment reduced 7-day re-open rate.
Compute a 95% confidence interval for the revenue lift and for the re-open-rate difference.
Estimate the required sample size per group to detect a 1.5 percentage point drop in re-open rate.
Give a launch recommendation and note key experimental risks.

Assumptions

Randomization is at the eligible rider level and stable over the 7-day window.
Revenue per rider is approximately normal by the central limit theorem at this sample size.
Re-open behavior is binary and independent across riders.

Sample Data

Example 1

Input

{"alpha":0.05,"power":0.8,"control_n":120000,"treatment_n":120000,"control_reopens":42388,"mde_reopen_drop":0.015,"treatment_reopens":40842,"control_sd_revenue":12.1,"baseline_reopen_rate":0.44,"control_mean_revenue":18.42,"treatment_sd_revenue":12.4,"treatment_mean_revenue":18.696,"treatment_ad_revenue_total":33154.5}

Output(none)

Your answer

Try one AI text evaluation on us

Get structured feedback, scored against a 4-axis rubric. Premium unlocks unlimited.

0 wordstarget ~200

Up next

LA/B Test for Lyft Ride Request ConversionMedium LTest In-App Ad PlacementMedium LPower Lyft Rider Promo TestHard

Next question