Scenario

You work on a digital audio subscription product and are testing a redesigned signup paywall that the team believes will increase trial starts by simplifying plan selection. The experiment is configured as a 50/50 split between control and treatment for new visitors landing on the signup flow. After 5 days, treatment appears to be outperforming control on trial-start rate, but you notice the observed allocation is 56/44 instead of the expected 50/50. You need to decide whether the result is trustworthy and what to do next.

Constraints

• Eligible traffic: 120,000 new signup-flow visitors per day
• Maximum experiment duration: 14 days
• Paid media spend cannot increase by more than 3% during the test
• Checkout error rate cannot rise above 1.2%
• Average page load time on the paywall cannot worsen by more than 150ms

Question

How would you design and analyze this experiment so that you can detect and respond appropriately to sample ratio mismatch, while still making a clear ship or no-ship decision if the test runs cleanly?