Rate Limiting an LLM API

Intercom's Fin AI Agent API must limit LLM usage per user to control token costs and maintain service stability. Implement a weighted sliding-window rate limiter that processes requests in chronological order.

Each request has a user identifier, timestamp, and estimated token cost. A request is accepted only when adding its cost would keep that user's accepted token usage within max_tokens during the active time window. Rejected requests do not consume capacity.

Formal Specification

Implement rate_limit(requests, window_seconds, max_tokens).

• requests is a list of [user_id, timestamp, token_cost] records, ordered by nondecreasing timestamp.
• user_id is a string.
• timestamp and token_cost are nonnegative integers.
• window_seconds and max_tokens are positive integers.
• Return a list of booleans, one per request, where True means accepted and False means rejected.
• For a request at time t, usage includes accepted requests with timestamps in (t - window_seconds, t].