Your question is Thread-Safe LLM Inference Cache. Start with the requirements on the right.
Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.
Implement a thread-safe least recently used cache for frequently repeated Intone Networks LLM inference results. The cache stores string keys and arbitrary result values, returns cached results, and evicts the least recently accessed entry when capacity is exceeded.
Your function receives a capacity and a sequence of operations. Each operation is either ['set', key, value] or ['get', key]. Return the values produced by get operations in order. Return None for a cache miss. A set operation marks the key as most recently used, whether the key is new or already exists.
The implementation must protect shared state with a synchronization primitive so simultaneous get and set calls cannot corrupt the cache. Both successful lookup and update, including eviction, should run in average O(1) time.
capacity, a positive integer, and operations, a list of operations. Keys are strings, and values may be any Python object.get operation, in operation order.def thread_safe_cache(capacity, operations):