Variance Unit Identification | Dataford Interview Questions

Your question is Variance Unit Identification. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

Problem

Finding the unit of variance

Asked in the Technical video round (40 min) stage. Third question, filed under 'Data Science Question'. Explain how you would determine the correct unit of variance for an online experiment, distinguishing the unit of randomization from the unit of analysis. Describe how repeated observations, clustering, and dependence affect variance and sample-size calculations.

State the assumptions and information you would request before choosing the unit.
Explain how your choice affects the primary metric, MDE, power, and required sample size.
Describe an analysis plan that avoids pseudoreplication and invalid standard errors.
Identify relevant experimentation pitfalls and give a concise decision rule for proceeding.

Your question is Variance Unit Identification. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

Finding the unit of variance

State the assumptions and information you would request before choosing the unit.
Explain how your choice affects the primary metric, MDE, power, and required sample size.
Describe an analysis plan that avoids pseudoreplication and invalid standard errors.
Identify relevant experimentation pitfalls and give a concise decision rule for proceeding.