314,552 interview questions from 6,000+ companies.
Identify major online experiment pitfalls and explain how they can bias results in a streaming product A/B test.
Design a marketing campaign experiment with a pre-registered metric plan, power calculation, and ship rule that respects guardrails.
Identify the main pitfalls that can distort A/B test interpretation and explain how to guard against them.
Explain how a primary metric differs from a guardrail metric and how both are used in A/B test decisions.
Explain why A/B testing matters in marketing analytics and how it supports causal, metric-driven campaign decisions.
Determine sample size and power for a customer survey or experiment, including MDE, guardrails, and a disciplined decision rule.
Design a landing-page A/B test with clear metrics, power, and significance criteria while guarding against common experiment pitfalls.
Investigate sample ratio mismatch and decide whether an experiment readout is trustworthy enough to ship.
Define the primary metric, guardrails, and power for a customer-facing A/B test before deciding whether to ship.
Estimate sample size and power for an experiment, define MDE and guardrails, and decide whether the test is worth running.
Design an experiment that accounts for novelty effects and network spillovers before deciding whether to ship.
Define primary and guardrail metrics for a discovery UI test, with power, MDE, and a pre-registered analysis plan.
Describe an A/B test you ran, what question it answered, how you measured success, and what you learned from the results.
Design and analyze an A/B test for a new email campaign, including metrics, power, guardrails, and common experiment risks.
Explain common online experimentation pitfalls and how to design, analyze, and decide in ways that avoid false wins.
Choose a primary success metric and guardrails for a game experiment, then explain how that choice drives power, analysis, and ship decisions.
Decide how to analyze an experiment when results are checked repeatedly and multiple comparisons may inflate false positives.
Design an experiment to determine whether a new product feature causes a meaningful retention lift without harming key guardrail metrics.
Explain how you would design and analyze a UX A/B test, from hypothesis and power to guardrails and launch decision.
Design an experiment to test whether a new marketing initiative improves conversion without harming key guardrails.
1,081 total questions
