Conversion Lift Test for Signup Feature | Dataford Interview Questions

Your question is Conversion Lift Test for Signup Feature. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

Business Context

StreamCart, a subscription video platform, launched a new signup page feature that shows personalized plan recommendations. The product team wants to know whether the feature improves paid conversion from visitor to subscriber.

Problem Statement

You are asked to design and analyze an experiment to determine whether the new feature increases conversion rate relative to the current signup flow.

Given Data

A 14-day user-level A/B test was run with random assignment at the first eligible visit.

Group	Users Exposed	Paid Conversions	Conversion Rate
Control (current signup flow)	52,400	6,131	11.70%
Treatment (personalized recommendations)	51,900	6,492	12.51%

Additional parameters:

Parameter	Value
Significance level	0.05
Test type	One-tailed
Baseline conversion rate for planning	11.7%
Minimum detectable effect	0.8 percentage points
Desired power	80%

Requirements

State the null and alternative hypotheses for this experiment.
Identify the correct unit of randomization and primary metric.
Compute the observed lift in conversion rate.
Run a two-proportion z-test and calculate the p-value.
Construct a 95% confidence interval for the treatment-control difference.
Determine whether the result is statistically significant at $\alpha = 0.05$ .
Check whether the observed sample size is roughly adequate for the stated minimum detectable effect.
Briefly list two guardrail metrics or experimental risks you would monitor before rollout.

Assumptions

Randomization was implemented correctly and independently across users.
Each user appears once in the analysis dataset.
No major tracking bugs or sample ratio mismatch occurred.
The experiment duration is long enough to cover day-of-week effects.