Batch vs Real-Time Pipelines

MediumPipelines00:00

Practice interviewer

In session

5 left

00:00

Whenever you're ready, walk me through how you'd approach this.

You need to log in / sign up to chat or submit.

Problem

Scenario

You're planning an operational workflow and need to choose the right pipeline pattern before implementation. Some workflows can tolerate delay, while others need immediate updates to drive downstream actions.

Question

How would you decide whether to use a batch process or a real-time process for an operational workflow?

What Drives the Choice

Freshness SLA and user impact
Tolerance for missing or late data
Operational complexity and on-call burden
Need for replay, correction, and auditability
Cost of always-on compute versus scheduled compute

Batch vs Real-Time

Batch

Simpler orchestration
Lower infrastructure cost
Better for heavy joins and recomputation
Higher latency

Real-Time

Low latency decisions
Higher operational complexity
More sensitive to schema and ordering issues
Better for immediate operational actions