Your question is Deduplicate Ordered Event Stream. Start with the requirements on the right.
Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.
In a Meta event ingestion pipeline, you receive a stream of event records in arrival order. Write a function that removes duplicate records while preserving the order of the first occurrence of each unique event.
A record is considered a duplicate if its event_id has already appeared earlier in the stream. Return the filtered list of records in the same relative order as their first appearance.
events, a list of event records"event_id"event_idevent_idevent_id values are hashabledef deduplicate_events(events):