Business Context

Splice is testing three homepage CTA variants on the Splice Sounds landing page to increase trial starts. Product wants to know whether to run a standard fixed-horizon A/B/n test or switch to a multi-armed bandit that reallocates traffic during the experiment.

Given Data

After the first 20,000 visitors, traffic was split evenly across three variants.

Additional planning inputs:

Problem Statement

Decide whether Splice should continue with a standard fixed-horizon experiment or switch to a multi-armed bandit. Your answer should quantify both the learning objective and the traffic-allocation objective.

Requirements

1. Compute the posterior mean conversion rate for each arm using a Beta(1,1) prior.
2. Estimate the probability each arm is best, and identify the current leader.
3. Calculate the expected number of incremental trial starts over the remaining 180,000 visitors if Splice:
   • keeps equal allocation across all 3 arms, or
   • sends all remaining traffic to the current best observed arm.
4. Compute the approximate per-variant sample size needed for a fixed-horizon test to detect a 1.5 percentage point lift from 12.0% to 13.5%.
5. Based on the numbers, recommend bandit vs standard A/B/n, and explain when each is preferable.

Assumptions

• Visitors are independent and randomly assigned in the initial phase.
• Conversion is binary: visitor starts a trial or not.
• No interference across users and no major seasonality shift during the month.
• For sample-size planning, use a normal approximation for two proportions and equal-sized groups.