How would you identify and handle missing or corrupted data points across a large distributed database using Python?
Translate this requirement into a PostgreSQL query over the provided source, expected measurement, and observation tables. Return every expected measurement that is missing or fails its validity checks.
expected_id, source_id, source_name, observed_at, observed_value, status, and issue_reason.source_id, then expected_id.| Column | Type | Description |
|---|---|---|
| source_idPK | INT | Identifier for a distributed data source |
| source_name | VARCHAR(100) | Human-readable source name |
| region | VARCHAR(50) | Deployment region of the source |
| Column | Type | Description |
|---|---|---|
| expected_idPK | INT | Identifier for an expected measurement point |
| source_id | INT | Expected source identifier |
| metric_name | VARCHAR(80) | Name of the expected metric |
| observed_at | TIMESTAMP | Expected observation timestamp |
| minimum_value | NUMERIC(10,2) | Minimum acceptable metric value |
| maximum_value | NUMERIC(10,2) | Maximum acceptable metric value |
| Column | Type | Description |
|---|---|---|
| observation_idPK | INT | Identifier for a received observation |
| expected_id | INT | Expected measurement associated with the observation |
| observed_value | NUMERIC(10,2) | Received metric value |
| quality_status | VARCHAR(20) | Quality classification supplied by ingestion |