Your question is TF-IDF From Raw Text Metadata. Take a moment with it on the right.
Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).
Write a function to parse, clean, and convert raw text metadata such as movie descriptions into a TF-IDF representation.
Implement a reusable Python pipeline that accepts raw movie metadata, handles missing or malformed text, normalizes the content, and returns a fitted vectorizer plus a sparse feature matrix. Explain preprocessing decisions such as HTML removal, tokenization, lowercasing, stop-word handling, lemmatization or stemming, n-gram selection, and vocabulary limits. Include a small example showing how a new description is transformed using the training vocabulary. The implementation should avoid data leakage by fitting IDF statistics only on training text and should include a basic validation or sanity check for the resulting representation.