Design a Survey

How would you design a survey?

Asked in the technical phone screen stage. Data intuition section, mostly statistics based. Treat the question as a practical experimentation exercise: explain how you would compare two survey designs or experiences, while making assumptions explicit because no population, baseline response rate, traffic volume, or effect size is provided.

State the hypothesis and choose a unit of randomization.
Define a primary metric, guardrails, secondary metrics, and an explicit MDE.
Show how you would calculate sample size and test duration once baseline inputs are available.
Pre-register the statistical analysis, including peeking, multiple comparisons, and SRM checks.
Give a ship, iterate, or do-not-ship rule and identify important validity risks.

How would you design a survey?

State the hypothesis and choose a unit of randomization.
Define a primary metric, guardrails, secondary metrics, and an explicit MDE.
Show how you would calculate sample size and test duration once baseline inputs are available.
Pre-register the statistical analysis, including peeking, multiple comparisons, and SRM checks.
Give a ship, iterate, or do-not-ship rule and identify important validity risks.

Problem

Design a Survey

Problem