Tell me about your experience with SQL and data warehouse concepts, including any real-time data pipeline scenarios you have worked on.
For the hands-on portion, use the supplied warehouse metadata, pipeline event, and load tables to produce a current pipeline freshness report for active datasets.
dataset_name, latest_event_time, latest_load_time, latency_seconds, and pipeline_state.latency_seconds descending with nulls last, then dataset_name ascending.| Column | Type | Description |
|---|---|---|
| dataset_idPK | INT | Unique dataset identifier |
| dataset_name | VARCHAR(100) | Display name of the dataset |
| source_system | VARCHAR(100) | System that produces the dataset |
| is_active | BOOLEAN | Whether the dataset is currently monitored |
| Column | Type | Description |
|---|---|---|
| event_idPK | INT | Unique pipeline event identifier |
| dataset_id | INT | Dataset associated with the event |
| event_time | TIMESTAMP | Time the pipeline event was recorded |
| event_type | VARCHAR(40) | Type of pipeline event |
| event_status | VARCHAR(20) | Outcome of the pipeline event |
| records_received | INT | Number of records received by the pipeline |
| Column | Type | Description |
|---|---|---|
| load_idPK | INT | Unique warehouse load identifier |
| dataset_id | INT | Dataset loaded into the warehouse |
| loaded_at | TIMESTAMP | Time the warehouse load completed or was attempted |
| load_status | VARCHAR(20) | Outcome of the warehouse load |
| records_loaded | INT | Number of records loaded successfully |