Business Context
StreamHub, a consumer video app, launched a new home-feed ranking model and saw higher daily active users (DAU) in the following two weeks. The product lead wants to know whether the observed increase is statistically meaningful or could plausibly be explained by normal day-to-day variation.
Problem Statement
Use the pre-launch and post-launch daily DAU data below to test whether mean DAU changed after the launch.
Given Data
| Period | Number of Days | Mean Daily DAU | Standard Deviation of Daily DAU |
|---|
| Pre-launch | 14 | 1,248,300 | 31,500 |
| Post-launch | 14 | 1,276,900 | 29,400 |
Additional inputs:
| Parameter | Value |
|---|
| Significance level | 0.05 |
| Test type | Two-sample, two-sided |
Requirements
- State the null and alternative hypotheses for mean daily DAU.
- Compute the standard error for the difference in means.
- Calculate the test statistic using a two-sample t-test.
- Approximate the degrees of freedom using Welch's method.
- Compute the two-sided p-value and determine whether the DAU change is statistically significant at the 5% level.
- Construct a 95% confidence interval for the change in mean DAU.
- Briefly explain whether the result is also practically meaningful for the business.
Assumptions
- Daily DAU observations within each period are treated as approximately independent.
- The pre-launch and post-launch windows are comparable aside from the launch.
- A Welch two-sample t-test is appropriate because the daily variances may differ.
- Ignore day-of-week adjustment for this exercise, but discuss it as a caveat in interpretation.
