A


Splice wants to improve activation for first-time users by replacing its current onboarding with a shorter guided flow. The product team ran a randomized A/B test on new signups and wants to know whether the new flow should be rolled out.
Use the experiment data below to determine whether the new onboarding flow increases 7-day activation rate for first-time users. Activation is defined as creating a project and exporting at least one audio clip within 7 days of signup.
| Group | Users | Activated within 7 days | Activation Rate |
|---|---|---|---|
| Control (old onboarding) | 18,420 | 4,421 | 24.00% |
| Treatment (new onboarding) | 18,615 | 4,748 | 25.51% |
Additional experiment settings:
| Parameter | Value |
|---|---|
| Significance level | 0.05 |
| Test type | Two-sided |
| Desired power | 0.80 |
| Baseline activation rate for planning | 0.240 |
| Minimum detectable effect for planning | 0.012 |
{"alpha":0.05,"control_n":18420,"treatment_n":18615,"control_rate":0.24,"mde_planning":0.012,"power_target":0.8,"treatment_rate":0.2551,"control_activated":4421,"treatment_activated":4748,"baseline_rate_planning":0.24}Output(none)