You work on an online learning product and your team wants to launch a new product change on a core student surface, such as a redesigned Chegg Study solution page or a new call-to-action in the subscription flow. The change is intended to improve student engagement or conversion, but there is concern that it could also hurt downstream experience or be misread if the experiment is not designed carefully.
How would you design an experiment to test this product change end to end? Walk through the hypothesis, success metrics, sample size and MDE, randomization choice, analysis plan, and the main risks you would watch for before making a ship decision.