Word Count on Large Files

MediumPython

Problem

Given 1TB of a file, how to check the word count using any language?

Implement count_words(chunks), where chunks is a list of string fragments read sequentially from the file. Return the number of whitespace-delimited words without loading the entire file into memory. A word may span multiple non-empty chunks, while whitespace and empty chunks end the current word.

Constraints

0 <= len(chunks)
Each chunk is a string and chunks appear in file order.
The total number of characters can be approximately 1TB.
Words are separated by characters for which Python isspace() returns True.
The complete file must not be materialized as one in-memory string.

Function Signature

def count_words(chunks):

CodePython 3

You need to log in / sign up to run or submit.Ln 2

Run your code to see test output here.