Your question is Databricks Skew and Shuffle Optimization. Take a moment with it on the right.
Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).
How do you optimize a Databricks notebook that is suffering from data skew and long shuffle operations? Explain how you would diagnose the bottleneck in the Spark UI and choose among Adaptive Query Execution, repartitioning, salting, broadcast joins, filtering, and pre-aggregation. Cover the trade-offs and explain which changes address skew versus general shuffle volume. Keep the answer focused on Apache Spark running in Databricks, using PySpark or Spark SQL.
