Module Instructor
In this course, you'll master how to design, run, and evaluate experiments that drive real product decisions. Learn from industry expertise how to bridge the gap between statistical rigor and business impact.
Banani brings 13+ years of experience leading data science initiatives across e-commerce, payments, and real estate. At Walmart, she has driven significant business impact through experimentation and causal inference:
Her expertise spans A/B testing, causal inference, growth experimentation, and AI/ML product development. She regularly presents to senior leadership and has spoken at industry conferences including MIT CODE, Grace Hopper Celebration, and IEEE GenAI Summit.
Why run experiments?
Phases in Experimentation
Defining the Problem Statement
Hypothesis Generation
Experimentation Details
Defining Test Metrics
Test Metrics Decision Flow
Decide on number of variations
Choosing test unit and assignment method
Qualified Target Base
Control Group Size & Structure
Exposure duration
Measurement horizon (Power Analysis)
Power Analysis Formula & Parameter Choices
Measurement horizon
Risk Tolerance and Ramp plan
Interpreting Results - P-Value
Misconceptions of P-Value
Interpreting Results- Confidence Intervals
Misconceptions of Confidence Interval
Statistical vs Practical Significance
Experiment Outcome Guideline
Stakeholder Communication Framework
Stakeholder Communication Expectations
Introduction to Casual Inference
Casual Inference Methods
Propensity Score Matching (PSM) Introduction
Propensity Score Matching (PSM) Core Idea
Propensity Score Matching (PSM) Methodology
Propensity Score Matching (PSM) Design
Propensity Score Marching (PSM) Real World Applications