Your question is Clean Missing and Corrupted Data. Start with the requirements and the three tables on the right.
Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.
How would you identify and handle missing or corrupted data points across a large distributed database using Python?
Translate this requirement into a PostgreSQL query over the provided source, expected measurement, and observation tables. Return every expected measurement that is missing or fails its validity checks.
expected_id, source_id, source_name, observed_at, observed_value, status, and issue_reason.source_id, then expected_id.| Column | Type | Description |
|---|---|---|
| source_idPK | INT | Identifier for a distributed data source |
| source_name | VARCHAR(100) | Human-readable source name |
| region | VARCHAR(50) | Deployment region of the source |
| Column | Type | Description |
|---|---|---|
| expected_idPK | INT | Identifier for an expected measurement point |
| source_id | INT | Expected source identifier |
| metric_name | VARCHAR(80) | Name of the expected metric |
| observed_at | TIMESTAMP | Expected observation timestamp |
| minimum_value | NUMERIC(10,2) | Minimum acceptable metric value |
| maximum_value | NUMERIC(10,2) | Maximum acceptable metric value |
| Column | Type | Description |
|---|---|---|
| observation_idPK | INT | Identifier for a received observation |
| expected_id | INT | Expected measurement associated with the observation |
| observed_value | NUMERIC(10,2) | Received metric value |
| quality_status | VARCHAR(20) | Quality classification supplied by ingestion |