Lazy Evaluation in PySpark | Dataford Interview Questions

Explain the concept of Lazy Evaluation in PySpark.

Implement a small Python pipeline that models this behavior using generators. The pipeline must apply filter_even, multiply, and take operations lazily, producing values only when the final result is consumed.

Input and Output

lazy_pipeline(data, operations) receives a list of integers and a list of operation objects. Each operation is a dictionary with type, and value when required. Return the resulting list of integers.

Constraints

0 <= len(data) <= 10^4
0 <= len(operations) <= 10
data contains integers between -10^9 and 10^9
Each operation type is one of filter_even, multiply, or take
multiply and take operations contain an integer value
take values are non-negative

Function Signature

def lazy_pipeline(data, operations):

Your question is Lazy Evaluation in PySpark. Start with the requirements in the Question tab.

Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.

Problem