Your question is Precision@K for Retrieval Evaluation. Start with the requirements on the right.
Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.
Pearson Revel ranks learning resources for each learner query. Given the ranked document IDs returned by the search system and the known relevant document IDs, compute Precision@K for every query and the mean Precision@K across all queries.
Precision@K is the number of relevant documents among the first K retrieved results divided by K. If fewer than K documents are returned, missing ranks count as non-relevant, so the denominator remains K.
Implement evaluate_precision_at_k(retrieved_results, relevant_results, k). retrieved_results and relevant_results are dictionaries mapping query IDs to lists of unique document ID strings. Every query in retrieved_results appears in relevant_results. k is a positive integer. Return a dictionary containing one floating-point Precision@K value for each query, plus a mean_precision value containing the arithmetic mean of the query scores.
def evaluate_precision_at_k(retrieved_results, relevant_results, k):