Exploratory vs Hypothesis-Driven Pricing Analysis

Business Context

StreamCart, a subscription video platform, saw a drop in weekly upgrade rate after changing its pricing page. A product manager wants to know whether the analyst should treat this as an exploratory analysis or a hypothesis-driven test.

Problem Statement

You are given two possible analysis setups based on the same business issue. One uses a pre-specified hypothesis and one starts by slicing the data into many segments after the fact. Decide which setup is hypothesis-driven, which is exploratory, and quantify why the distinction matters statistically.

Given Data

For Scenario A, assume the observed counts are 816 upgrades before and 712 upgrades after.

Requirements

1. Classify Scenario A and Scenario B as exploratory or hypothesis-driven.
2. For Scenario A, state the null and alternative hypotheses.
3. Conduct a two-proportion z-test for Scenario A.
4. Compute a 95% confidence interval for the change in upgrade rate in Scenario A.
5. For Scenario B, calculate the family-wise probability of at least one false positive across 20 independent tests at $\alpha = 0.05$.
6. Explain how the statistical evidence should change the analyst's reporting and next step.

Assumptions

• Users in the before and after groups are independent for Scenario A.
• The hypothesis in Scenario A was defined before looking at the results.
• The 20 tests in Scenario B are treated as independent for the false-positive calculation.
• Use a two-sided test for Scenario A.