Word Count on Large Files | Dataford Interview Questions

Given 1TB of a file, how to check the word count using any language?

Implement count_words(chunks), where chunks is a list of string fragments read sequentially from the file. Return the number of whitespace-delimited words without loading the entire file into memory. A word may span multiple non-empty chunks, while whitespace and empty chunks end the current word.

Constraints

0 <= len(chunks)
Each chunk is a string and chunks appear in file order.
The total number of characters can be approximately 1TB.
Words are separated by characters for which isspace() returns True.
The complete file must not be materialized as one in-memory string.

Function Signature

def count_words(chunks):

Your question is Word Count on Large Files. Start with the requirements in the Question tab.

Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.

Problem

Given 1TB of a file, how to check the word count using any language?

Constraints

0 <= len(chunks)
Each chunk is a string and chunks appear in file order.
The total number of characters can be approximately 1TB.
Words are separated by characters for which isspace() returns True.
The complete file must not be materialized as one in-memory string.

Function Signature

def count_words(chunks):

Your question is Word Count on Large Files. Start with the requirements in the Question tab.

Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.