
Meta’s event pipeline for Facebook activity can receive the same logical event multiple times, and some duplicates arrive late. Write a SQL query to return one canonical row per event_id, keeping the most recently ingested record.
event_id.event_id across the full table, even if duplicates arrive on different ingestion dates.ingested_at for each event_id.event_id have the same ingested_at, keep the one with the greater record_id.event_id, user_id, event_name, event_time, and ingested_at.event_id ascending.| column | type | description |
|---|---|---|
| record_id | INT | Unique row identifier for each ingested record |
| event_id | VARCHAR(50) | Logical event identifier used for deduplication |
| user_id | INT | Meta user identifier |
| event_name | VARCHAR(50) | Event type |
| event_time | TIMESTAMP | Time the event actually occurred |
| ingested_at | TIMESTAMP | Time the record was received by the warehouse |
| source_surface | VARCHAR(50) | Meta surface that produced the event |
| record_id | event_id | user_id | event_name | event_time | ingested_at | source_surface |
|---|---|---|---|---|---|---|
| 103 | evt_1003 | 12 | like | 2024-04-01 09:02:00 | 2024-04-01 09:02:10 | |
| 101 | evt_1001 | 10 | impression | 2024-04-01 09:00:00 | 2024-04-01 09:00:05 | |
| 108 | evt_1005 | 14 | click | 2024-04-01 09:05:00 | 2024-04-01 09:05:20 | Reels |
| 110 | evt_1006 | 15 | comment | 2024-04-01 09:06:00 | 2024-04-01 09:06:10 | |
| 112 | evt_1007 | 16 | share | 2024-04-01 09:07:00 | 2024-04-01 09:07:05 |
| event_id | user_id | event_name | event_time | ingested_at |
|---|---|---|---|---|
| evt_1001 | 10 | impression | 2024-04-01 09:00:00 | 2024-04-01 09:10:00 |
| evt_1002 | 11 | click | 2024-04-01 09:01:00 | 2024-04-01 09:01:03 |
| evt_1003 | 12 | like | 2024-04-01 09:02:00 | 2024-04-01 09:12:00 |
| evt_1004 | 13 | impression | 2024-04-01 09:03:00 | 2024-04-01 09:03:05 |
| evt_1005 | 14 | click | 2024-04-01 09:05:00 | 2024-04-01 09:05:20 |
| evt_1006 | 15 | comment | 2024-04-01 09:06:00 | 2024-04-01 09:06:10 |
| evt_1007 | 16 | share | 2024-04-01 09:07:00 | 2024-04-01 09:07:05 |
| Column | Type | Description |
|---|---|---|
| record_idPK | INT | Unique identifier for each ingested row |
| event_id | VARCHAR(50) | Logical event identifier used to detect duplicates |
| user_id | INT | User associated with the event |
| event_name | VARCHAR(50) | Type of event generated on a Meta surface |
| event_time | TIMESTAMP | Timestamp when the event occurred |
| ingested_at | TIMESTAMP | Timestamp when the event record arrived in the warehouse |
| source_surface | VARCHAR(50) | Meta product surface that emitted the event |
| record_id | event_id | user_id | event_name | event_time | ingested_at | source_surface |
|---|---|---|---|---|---|---|
| 103 | evt_1003 | 12 | like | 2024-04-01 09:02:00 | 2024-04-01 09:02:10 | |
| 101 | evt_1001 | 10 | impression | 2024-04-01 09:00:00 | 2024-04-01 09:00:05 | |
| 108 | evt_1005 | 14 | click | 2024-04-01 09:05:00 | 2024-04-01 09:05:20 | Reels |
| 110 | evt_1006 | 15 | comment | 2024-04-01 09:06:00 | 2024-04-01 09:06:10 | |
| 112 | evt_1007 | 16 | share | 2024-04-01 09:07:00 | 2024-04-01 09:07:05 | |
| 102 | evt_1002 | 11 | click | 2024-04-01 09:01:00 | 2024-04-01 09:01:03 | |
| 104 | evt_1001 | 10 | impression | 2024-04-01 09:00:00 | 2024-04-01 09:10:00 | |
| 105 | evt_1003 | 12 | like | 2024-04-01 09:02:00 | 2024-04-01 09:12:00 | |
| 106 | evt_1004 | 13 | impression | 2024-04-01 09:03:00 | 2024-04-01 09:03:05 | Messenger |
| 109 | 17 | click | 2024-04-01 09:08:00 | 2024-04-01 09:08:01 | ||
| 107 | evt_1004 | 13 | impression | 2024-04-01 09:03:00 | 2024-04-01 09:03:05 | Messenger |
| 111 | evt_1006 | 15 | comment | 2024-04-01 09:05:59 | ||
| 1 | evt_1002 | 40 | hover | 2024-04-03 05:12:22 | 2024-03-30 21:50:33 | null |
| 2 | evt_1019 | 39 | reaction | 2024-04-02 13:53:09 | 2024-04-01 07:25:44 | Instagram Explore |
| 3 | evt_1009 | 13 | view | 2024-04-04 07:56:24 | 2024-04-02 17:40:54 | Portal |
| 4 | evt_1004 | 63 | comment | 2024-03-30 06:28:32 | 2024-03-31 13:08:59 | |
| 5 | evt_1002 | 75 | scroll | 2024-03-31 00:05:29 | 2024-04-02 19:37:02 | Feed |
| 6 | evt_1021 | 64 | logout | 2024-04-03 15:38:54 | 2024-04-01 01:12:59 | Reels |
| 7 | evt_1009 | 57 | signup | 2024-03-31 05:03:18 | 2024-04-03 19:49:17 | Reels |
| 8 | evt_1025 | 66 | scroll | 2024-03-30 07:21:30 | 2024-03-31 14:15:53 | Workplace |
| 9 | evt_1017 | 97 | unfollow | 2024-04-03 11:15:39 | 2024-03-29 10:37:27 | Creator Studio |
| 10 | evt_1003 | 82 | like | null | 2024-04-02 23:56:44 | Instagram Explore |
| 11 | evt_1005 | 26 | submit | 2024-03-31 19:02:45 | 2024-04-02 06:46:08 | Watch |
| 12 | null | 23 | signup | 2024-03-29 19:56:45 | 2024-03-29 20:59:51 | Gaming |
| 13 | evt_1026 | 19 | purchase | 2024-03-29 10:49:35 | 2024-04-01 23:03:53 | null |
| 14 | evt_1017 | 71 | play | 2024-04-04 05:41:27 | 2024-04-03 22:43:22 | Marketplace |
| 15 | evt_1016 | 25 | submit | null | 2024-04-02 21:29:38 | Pages |
| 16 | evt_1003 | 43 | search | 2024-04-01 08:59:26 | 2024-04-02 07:49:12 |
| event_id | user_id | event_name | event_time | ingested_at |
|---|---|---|---|---|
| evt_1001 | 10 | impression | 2024-04-01 09:00:00 | 2024-04-01 09:10:00 |
| evt_1002 | 75 | scroll | 2024-03-31 00:05:29 | 2024-04-02 19:37:02 |
| evt_1003 | 82 | like | null | 2024-04-02 23:56:44 |
| evt_1004 | 13 | impression | 2024-04-01 09:03:00 | 2024-04-01 09:03:05 |
| evt_1005 | 26 | submit | 2024-03-31 19:02:45 | 2024-04-02 06:46:08 |
| evt_1006 | 15 | comment | 2024-04-01 09:06:00 | 2024-04-01 09:06:10 |
| evt_1007 | 16 | share | 2024-04-01 09:07:00 | 2024-04-01 09:07:05 |
| evt_1009 | 57 | signup | 2024-03-31 05:03:18 | 2024-04-03 19:49:17 |
| evt_1016 | 25 | submit | null | 2024-04-02 21:29:38 |
| evt_1017 | 71 | play | 2024-04-04 05:41:27 | 2024-04-03 22:43:22 |
| evt_1019 | 39 | reaction | 2024-04-02 13:53:09 | 2024-04-01 07:25:44 |
| evt_1021 | 64 | logout | 2024-04-03 15:38:54 | 2024-04-01 01:12:59 |
| evt_1025 | 66 | scroll | 2024-03-30 07:21:30 | 2024-03-31 14:15:53 |
| evt_1026 | 19 | purchase | 2024-03-29 10:49:35 | 2024-04-01 23:03:53 |