


StreamHub is launching a new personalized re-engagement feature intended to improve 30-day user retention. The product team wants an experiment design that can both detect a meaningful lift and support a rollout decision.
Design and analyze a randomized A/B test to measure whether the feature improves 30-day retention. Use the baseline retention and target lift below to size the experiment, then use the observed experiment results to test whether the feature worked.
| Metric | Value |
|---|---|
| Baseline 30-day retention | 24.0% |
| Minimum detectable absolute lift | 1.8 percentage points |
| Significance level | 0.05 |
| Desired power | 80% |
| Control users observed | 18,400 |
| Control retained at day 30 | 4,324 |
| Treatment users observed | 18,650 |
| Treatment retained at day 30 | 4,610 |
Assume a two-sided test and equal traffic allocation during planning.
{"alpha":0.05,"power":0.8,"control_n":18400,"treatment_n":18650,"control_retained":4324,"treatment_retained":4610,"baseline_retention_rate":0.24,"minimum_detectable_effect_absolute":0.018}Output(none)