Business Context

StreamCart, a subscription video platform, tested a new signup page intended to increase trial starts. Before acting on the result, the team wants to know whether the experiment is valid and whether the observed lift is trustworthy.

Problem Statement

You are given the experiment results and a few diagnostics. Determine whether the A/B test can be interpreted causally, which assumptions are required for validity, and whether the measured lift is statistically significant.

Given Data

Additional test settings:

Requirements

1. State the null and alternative hypotheses for the primary metric.
2. Test whether the conversion lift is statistically significant using a two-proportion z-test.
3. Compute a 95% confidence interval for the difference in conversion rates.
4. Check for sample ratio mismatch against the planned 50/50 allocation.
5. List the key assumptions required for an A/B test to be valid.
6. Based on the diagnostics, explain whether the result is fully trustworthy and what you would do next.

Assumptions

• User-level randomization was intended.
• Trial start is measured once per assigned user.
• Use a two-sided test.
• Ignore finite population corrections.