An Instagram Reels ranking team uses a simple Bernoulli randomizer in an internal experiment to decide whether an eligible impression gets a new ranking policy or the baseline policy. Before trusting downstream AARRR funnel reads on metrics like IG Save, the team wants to verify that the randomizer behaves like a fair coin. A Sample Ratio Mismatch (SRM) alert was triggered because treatment assignment looked slightly imbalanced.
Treat each assignment as a coin flip: Heads = treatment, Tails = control. You suspect the coin is biased away from 50/50. Test this statistically using the observed assignments.
| Metric | Value |
|---|---|
| Total assignment events | 10,000 |
| Treatment assignments (Heads) | 5,230 |
| Control assignments (Tails) | 4,770 |
| Expected treatment probability under fair coin | 0.50 |
| Significance level | 0.05 |
Assume assignments are independent and the logging pipeline is correct. Ignore CUPED here because this is a randomization-validity check, not an outcome analysis.