PySpark Transformations | Dataford Interview Questions

What PySpark transformations have you implemented in your recent projects?

Implement the equivalent transformation pipeline as transform_records(records): filter records with status == "completed" and amount > 0, aggregate total amount and record count by category, then return results sorted by category.

Input records are dictionaries with category, amount, and status. Return dictionaries containing category, total_amount, and count.

Example: [{'category':'A','amount':10,'status':'completed'}] returns [{'category':'A','total_amount':10,'count':1}].

Constraints

0 <= len(records) <= 100000
Each record contains category, amount, and status
category is a non-empty string
amount is an integer or floating-point number
status is a string

Function Signature

def transform_records(records):

Your question is PySpark Transformations. Start with the requirements in the Question tab.

Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.

Problem