Apache Kafka Consumer Pseudocode | Dataford Interview Questions

Write pseudocode for a Kafka consumer application.

Asked in the Technical interview (live coding) stage. Implement a deterministic simulation of one consumer partition that polls records in batches, retries failed processing, and commits an offset only after successful processing.

Input and Output

Implement consume_messages(records, committed_offset, batch_size, max_retries). Each record is a dictionary with offset, value, and failures_before_success. Return { "processed": [...], "committed_offset": ... }. Skip records below the committed offset. Stop at the first record that still fails after all retries, without processing later records.

Constraints

Records are ordered by increasing offset.
0 <= len(records) <= 1000.
0 <= failures_before_success <= 10.
1 <= batch_size <= 1000.
0 <= max_retries <= 10.
Each record contains integer offset, any value, and integer failures_before_success.

Function Signature

def consume_messages(records, committed_offset, batch_size, max_retries):

Your question is Apache Kafka Consumer Pseudocode. Start with the requirements in the Question tab.

Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.

Problem