Dataford
Interview QuestionsInterview GuidesExperiencesMock InterviewsPricing
Get started

Clean and Normalize Data Streams

HardPython00:00
Practice interviewer
In session
5 left
00:00

Your question is Clean and Normalize Data Streams. Start with the requirements on the right.

Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.

You need to log in / sign up to run or submit.

Problem

Amgen's laboratory systems can emit sample events with inconsistent field names, capitalization, whitespace, timestamp formats, and numeric units. Implement normalize_stream to clean a batch of incoming event records and return only valid, canonical events.

Formal Specification

Input is a list of dictionaries. Each record may use these aliases:

  • Event ID: event_id or id
  • Sample ID: sample_id or sample or specimen_id
  • Status: status or state
  • Dose: dose_mg or dose
  • Timestamp: timestamp or time

Return a list of dictionaries with exactly these keys: event_id, sample_id, status, dose_mg, and timestamp.

Normalize values as follows:

  1. Trim identifier whitespace and convert identifiers to strings.
  2. Convert statuses to lowercase and map complete and completed to completed, in-progress and processing to in_progress, and failed and error to failed.
  3. Parse doses such as "12.5 mg", "1,000", or numeric values into nonnegative floats. Missing doses become None.
  4. Parse ISO-8601 timestamps. Treat timestamps without a timezone as UTC and return UTC timestamps ending in Z.
  5. Discard records missing required fields, having unknown statuses, invalid timestamps, or invalid doses.
  6. If multiple valid records share an event_id, keep the last valid record while preserving the order of each event's first appearance.

Constraints

  • 1 <= len(records) <= 10^5
  • Each record has at most 20 fields
  • Identifiers contain at most 100 characters
  • Timestamps are ISO-8601 strings, datetime values, or invalid input
  • Statuses are case-insensitive and may contain spaces, underscores, or hyphens

Function Signature

def normalize_stream(records):
Your solutionPython 3
You need to log in / sign up to run or submit.
Run your code to see test output