Context

In distributed systems interviews, Spark dependency types are used to test whether you understand execution planning, shuffles, and performance trade-offs.

Question

Explain the difference between narrow and wide dependencies in Spark.

Your answer should cover:

1. What each dependency type means at the partition level
2. Which operations typically create each type of dependency
3. Why wide dependencies usually trigger shuffles and are more expensive
4. How dependency type affects fault tolerance, stage boundaries, and performance tuning

Scope Guidance

The interviewer expects a systems-oriented explanation rather than Spark API memorization. You should define both terms clearly, compare them directly, and connect them to execution behavior such as stage splitting, data movement, and recovery after failure. Brief examples using common transformations like map, filter, reduceByKey, or join are enough.