How would you optimize a data processing pipeline so it scales efficiently without consuming excessive compute resources?
