Explain how PySpark manages memory and partition sizes when executing large-scale data transformations.
