LLM API Rate Limiting

MediumPython

Problem

Implement a rate limiter for the Axis Max Life Insurance policy-assistance LLM endpoint. Each user may make at most limit accepted requests during any rolling interval of window seconds.

Process requests in the order received. For each request, remove that user's accepted request timestamps that are outside the current window. Accept the request if fewer than limit accepted requests remain, otherwise reject it. Rejected requests do not consume quota.

Formal Specification

Implement rate_limit(requests, limit, window).

requests is a list of two-element lists, [user_id, timestamp], ordered by nondecreasing timestamp.
user_id is a string.
timestamp, limit, and window are positive integers.
Return a list of booleans in request order. True means accepted and False means rejected.
A request at time t considers timestamps in the half-open interval [t - window, t) expired. A request at exactly t - window is therefore expired.

Constraints

1 <= len(requests) <= 2 * 10^5
1 <= limit, window <= 10^9
0 <= timestamp <= 10^9
user_id is a non-empty string
Requests are ordered by nondecreasing timestamp

Function Signature

def rate_limit(requests, limit, window):

Implement a rate limiter for the Axis Max Life Insurance policy-assistance LLM endpoint. Each user may make at most limit accepted requests during any rolling interval of window seconds.

Formal Specification

Implement rate_limit(requests, limit, window).

requests is a list of two-element lists, [user_id, timestamp], ordered by nondecreasing timestamp.
user_id is a string.
timestamp, limit, and window are positive integers.
Return a list of booleans in request order. True means accepted and False means rejected.
A request at time t considers timestamps in the half-open interval [t - window, t) expired. A request at exactly t - window is therefore expired.