Transform Nested JSON at Scale | Dataford Interview Questions

Write a Python or PySpark script to clean, transform, and aggregate a massive dataset containing complex nested JSON structures.

For this Python version, implement process_nested_events(records). Each record contains a user.id and an events array. Each event contains id, type, value, and metadata.region. Ignore malformed records or events, normalize text fields by trimming and lowercasing, and deduplicate events globally by event ID. Return records grouped by region and event type, sorted by both fields, with count and total_value.

Input is a list of dictionaries. Output is a list of dictionaries with keys region, event_type, count, and total_value.

Constraints

1 <= len(records) <= 5000
Each record may contain zero or more events
Event IDs are hashable and represent global event identity
Valid event values are integers or floating-point numbers, excluding booleans
Malformed records and events must be ignored
Output groups must be sorted lexicographically by region, then event_type

Function Signature

def process_nested_events(records):

Your question is Transform Nested JSON at Scale. Start with the requirements in the Question tab.

Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.

Problem

Write a Python or PySpark script to clean, transform, and aggregate a massive dataset containing complex nested JSON structures.

Input is a list of dictionaries. Output is a list of dictionaries with keys region, event_type, count, and total_value.

Constraints

1 <= len(records) <= 5000
Each record may contain zero or more events
Event IDs are hashable and represent global event identity
Valid event values are integers or floating-point numbers, excluding booleans
Malformed records and events must be ignored
Output groups must be sorted lexicographically by region, then event_type

Function Signature

def process_nested_events(records):

Your question is Transform Nested JSON at Scale. Start with the requirements in the Question tab.

Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.