Your question is State Management in Data Pipelines. Take a moment with it on the right.
Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).
You're working on a distributed data pipeline and need a clear way to think about how state is tracked across ingestion, processing, and recovery. The pipeline may run in batch or streaming mode, and multiple workers may process related records at different times.
How do you handle state management in distributed applications?