NovaAssist is testing a new AI reply-suggestion feature in its customer support product. Traffic is limited because only a small share of agents currently use the beta workflow, so the team needs to know whether a standard A/B test is feasible before launch.
Estimate the minimum sample size needed to detect a meaningful improvement in adoption rate, then assess whether the available traffic is enough to run the experiment in a reasonable time.
The primary metric is agent adoption rate: the share of eligible sessions where the agent accepts at least one AI suggestion.
| Metric | Value |
|---|---|
| Baseline adoption rate | 18.0% |
| Minimum detectable effect (absolute lift) | 3.0 percentage points |
| Target treatment adoption rate | 21.0% |
| Significance level | 0.05 |
| Desired power | 80% |
| Eligible sessions per week | 8,400 |
| Planned traffic split | 50% control / 50% treatment |