Deduplicating Records in Code | Dataford Interview Questions

Thornburg Investment receives records from multiple portfolio holdings feeds. Implement a function that removes duplicate records according to one or more identifying fields while preserving the order of their first appearances.

Formal Specification

Given records, a list of dictionaries, and key_fields, a list of dictionary keys, return a new list containing only the first record for each unique combination of key_fields. The identifying field values are hashable, and every record contains every requested key. Do not modify the input list or its dictionaries.

Two records are duplicates when their values match for every field in key_fields, regardless of differences in other fields. Preserve the original dictionary objects in the returned list.

Constraints

0 <= len(records) <= 100,000
1 <= len(key_fields) <= 10
Each key-field value is hashable
Every requested field exists in every record

Function Signature

def deduplicate_records(records, key_fields):

Your question is Deduplicating Records in Code. Start with the requirements in the Question tab.

Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.

Problem