PySpark Word Count | Dataford Interview Questions

Can you write a word count program in PySpark?

Asked in the technical interview stage. Implement the core word-count transformation as a single Python function using standard-library behavior so it can be graded without a Spark installation.

Input and Output

lines is a list of strings. Return a dictionary mapping each lowercase alphanumeric word to its frequency. Extract words with the pattern [A-Za-z0-9]+; punctuation separates words.

Constraints

0 <= len(lines) <= 10^4
Each input element is a string
Words contain only ASCII letters and digits
Punctuation separates words and is not included in the output

Function Signature

def word_count(lines):

Your question is PySpark Word Count. Start with the requirements in the Question tab.

Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.

Problem