Your question is Real-Time Data Ingestion Pipeline. Take a moment with it on the right.
Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).
How would you design a real-time data ingestion pipeline that processes terabytes of data from highly fragmented, external APIs for Peregrine Technologies?
Explain how you would standardize incompatible API contracts, handle pagination and rate limits, support incremental extraction, and preserve source data for replay. Cover ingestion, transformation, storage, orchestration, data quality, observability, failure recovery, and backfills. Include the trade-offs between batch polling, asynchronous extraction, and streaming, plus how you would make processing idempotent.