Optimize Search for Distributed Data | Dataford Interview Questions

How would you optimize a search algorithm for a massive, distributed dataset? Assume the dataset is represented as sorted shards, where each record is [key, value]. Implement search_distributed(shards, target, worker_count) to return every record whose key equals target, preserving shard order and record order; use shard-range pruning, binary search, and parallel shard searches where useful.

Constraints

1 <= len(shards) <= 100000
The total number of records across all shards is at most 10000000
Each nonempty shard is sorted by its first record element
Keys and values are integers or strings, and keys are mutually comparable
1 <= worker_count <= 1000
Duplicate keys may occur within or across shards
Return all matching records in original shard order

Function Signature

def search_distributed(shards, target, worker_count):

Your question is Optimize Search for Distributed Data. Start with the requirements in the Question tab.

Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.

Problem

Constraints

1 <= len(shards) <= 100000
The total number of records across all shards is at most 10000000
Each nonempty shard is sorted by its first record element
Keys and values are integers or strings, and keys are mutually comparable
1 <= worker_count <= 1000
Duplicate keys may occur within or across shards
Return all matching records in original shard order

Function Signature

def search_distributed(shards, target, worker_count):

Your question is Optimize Search for Distributed Data. Start with the requirements in the Question tab.

Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.