Merge and Deduplicate Retrieval Results

MediumPython

Problem

T. Rowe Price's research search experience receives ranked document results from multiple retrieval sources. Merge these results, keep only the best occurrence of each document, and return the highest-ranked documents in a deterministic order.

Implement merge_results(results, source_priority, top_k).

Each record contains doc_id, score, title, and source. A document's best occurrence is the record with the highest score. If scores tie, prefer the source with the lower priority value. If both score and priority tie, prefer the occurrence encountered first. Return the selected records sorted by descending score, ascending source priority, and ascending encounter order. Return at most top_k records.

Formal Specification

results is a list of result lists. Each inner list is ordered, but its order does not determine the final ranking.
source_priority is a dictionary mapping every source name to an integer priority.
top_k is a nonnegative integer.
Return a list of the original record dictionaries, with no duplicate doc_id values.

Constraints

1 <= len(results) <= 100
0 <= sum(len(batch) for batch in results) <= 10^5
0 <= top_k <= 10^5
Document IDs are non-empty strings
Every record source exists in source_priority
Scores are numeric

Function Signature

def merge_results(results, source_priority, top_k):

Implement merge_results(results, source_priority, top_k).

Formal Specification

results is a list of result lists. Each inner list is ordered, but its order does not determine the final ranking.
source_priority is a dictionary mapping every source name to an integer priority.
top_k is a nonnegative integer.
Return a list of the original record dictionaries, with no duplicate doc_id values.