How would you design a data processing pipeline for large, varied datasets coming from multiple platforms?