Your question is Exponential Backoff with Jitter. Start with the requirements on the right.
Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.
Implement a deterministic retry policy for calls to the OpenAI API. The input provides the result returned by each hypothetical API attempt, allowing the retry algorithm to be tested without making network requests.
Retry responses with HTTP status 408, 409, 429, or any status from 500 through 599. Return immediately for a successful status from 200 through 299 or for a non-retryable error. Stop after max_retries retries, meaning the function may process at most max_retries + 1 outcomes.
For retry number r, where the first retry has r = 0, calculate the capped exponential delay as min(max_delay, base_delay * 2^r). Add deterministic jitter equal to jitter_values[r] * 0.25 * capped_delay, where every jitter value is between 0 and 1.
Return a dictionary containing status, attempts, and delays. Successful calls include the response value; failures include the final error.
outcomes is a non-empty list of dictionaries with an integer status and optional value or error. max_retries, base_delay, and max_delay are non-negative numbers. jitter_values supplies one value for each possible retry. Return a dictionary.
def retry_openai_call(outcomes, max_retries, base_delay, max_delay, jitter_values):