Precision@K for Retrieval Evaluation | Dataford Interview Questions

Pearson Revel ranks learning resources for each learner query. Given the ranked document IDs returned by the search system and the known relevant document IDs, compute Precision@K for every query and the mean Precision@K across all queries.

Precision@K is the number of relevant documents among the first K retrieved results divided by K. If fewer than K documents are returned, missing ranks count as non-relevant, so the denominator remains K.

Formal Specification

Implement evaluate_precision_at_k(retrieved_results, relevant_results, k). retrieved_results and relevant_results are dictionaries mapping query IDs to lists of unique document ID strings. Every query in retrieved_results appears in relevant_results. k is a positive integer. Return a dictionary containing one floating-point Precision@K value for each query, plus a mean_precision value containing the arithmetic mean of the query scores.

Constraints

1 <= number of queries <= 10^4
1 <= k <= 10^5
Each query may return at most 10^5 document IDs
Document IDs are non-empty strings
Document IDs within each ranked result list are unique
Every query in retrieved_results appears in relevant_results

Function Signature

def evaluate_precision_at_k(retrieved_results, relevant_results, k):

Your question is Precision@K for Retrieval Evaluation. Start with the requirements in the Question tab.

Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.

Problem

Formal Specification

Constraints

1 <= number of queries <= 10^4
1 <= k <= 10^5
Each query may return at most 10^5 document IDs
Document IDs are non-empty strings
Document IDs within each ranked result list are unique
Every query in retrieved_results appears in relevant_results

Function Signature

def evaluate_precision_at_k(retrieved_results, relevant_results, k):

Your question is Precision@K for Retrieval Evaluation. Start with the requirements in the Question tab.

Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.