Recipes

Handle rate limits and retries

Build a request wrapper that honors Retry-After, backs off with a random wait that grows with each attempt, caps attempts and total time, and repeats only safe calls. Use it when several of your workers share one organization, when a backfill reads thousands of audits, or when 429 responses show up in your logs. Your organization can send a burst of 100 requests, then 5 per second, shared by every key; the sandbox has a separate budget with the same limits.

Prerequisites

  • An access token in $BRELLIUM_ACCESS_TOKEN, cached and reused (Authenticate with OAuth2 client credentials).
  • An audit_id, for example from GET /v2/audits?limit=1. This page uses the sample audit a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d.
  • A TypeScript runtime such as Bun, or Python 3.10 or later with requests.
  • With a live key the response contains protected health information (PHI); a sandbox key returns the sample audit.

Steps

Read the headers on a normal response

Every authenticated /v2 response carries the state of your organization's token bucket, so you can slow down before the first 429:

curl --include \
  "https://api.brellium.com/v2/audits/a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d" \
  --header "Authorization: Bearer $BRELLIUM_ACCESS_TOKEN"

const res = await fetch(
  "https://api.brellium.com/v2/audits/a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d",
  {
    headers: { Authorization: `Bearer ${process.env.BRELLIUM_ACCESS_TOKEN}` },
  },
);
console.log(`HTTP ${res.status}`);
for (const [name, value] of res.headers) console.log(`${name}: ${value}`);
if (!res.ok)
  throw new Error(
    `HTTP ${res.status}: ${JSON.stringify((await res.json()).error)}`,
  );
console.log(await res.json());

import BrelliumClient from "@brellium/api-client";

const client = new BrelliumClient({ oAuth2: process.env.BRELLIUM_ACCESS_TOKEN });

const { data, response } = await client.audits
  .retrieve("a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d")
  .withResponse();
console.log(`HTTP ${response.status}`);
for (const [name, value] of response.headers) console.log(`${name}: ${value}`);
console.log(data);

import os

from brellium_api_client import BrelliumClient

client = BrelliumClient(o_auth2=os.environ["BRELLIUM_ACCESS_TOKEN"])

response = client.audits.with_raw_response.retrieve(
    "a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d"
)
print(f"HTTP {response.status_code}")
for name, value in response.headers.items():
    print(f"{name}: {value}")
print(response.parse().to_json())

HTTP/2 200
content-type: application/json
ratelimit-limit: 100
ratelimit-remaining: 99
ratelimit-reset: 0
x-request-id: 7f0b3c1e-2a4d-4b8e-9c11-5d6e7f8a9b0c

{
  "audit_id": "a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d",
  "object": "audit",
  "status": "complete",
  "flags": ["Failed", "Missing Signature"],
  "display_info": [
    /* ... */
  ], // display_info, people and responses as in Retrieve an audit by id
  "people": [ /* ... */ ],
  "responses": [ /* ... */ ]
}

Only the headers matter here; the body is the sample audit (Retrieve an audit by id). Over HTTP/2 the header names arrive in lowercase.

RateLimit-Remaining is the number of requests you can still send right now. RateLimit-Reset is 0 on every allowed response, so pace your requests by RateLimit-Remaining. A 401 carries none of these headers and does not count against the rate limit, because the API checks the access token first.

Decide what is safe to retry

A 429 is always safe to retry, because a refused request changes nothing and spends no token from the bucket. A 400 or 404 does spend one. After a 5xx or a dropped connection you cannot tell whether the request ran, so retry only when a repeat does no harm.

Call After a 5xx or a dropped connection
Any GET or HEAD Retry.
PATCH /v2/audits/{audit_id}, POST /v2/audits/batch_update Retry with the same body: flags replaces the audit's flags, so a repeat leaves the same flags.
POST /v2/documents Retry with the same body only when every document has an external_id. The repeat then returns the documents the first call created, with a new upload URL for each file that has not arrived yet, and creates nothing. Without external_id values a repeat creates a second batch (Upload documents).
POST /v2/users Do not retry automatically. A repeat after a success returns 409 INVITATION_PENDING, or 409 USER_EXISTS once the user has signed in to Brellium, so treat the first call as done. Retry only after a 502 INVITE_FAILED: the user is kept, and the repeat sends the invitation. A retry creates no second user.
DELETE /v2/users/{user_id} Retry. A repeat after a success returns 404, so treat it as done. A 502 REVOKE_FAILED keeps the user, so call again.
POST /v2/email_recipients Do not retry automatically. A repeat after a success returns 409 EMAIL_RECIPIENT_EXISTS; fetch the email recipient with GET /v2/email_recipients?email=. Lists can take a few seconds to include a new recipient.
POST /v2/oauth/token Retry a 502 BAD_GATEWAY (Token endpoint temporarily unavailable) or any other 5xx with backoff. A mint changes no records, but each attempt counts against the token endpoint's limit (a burst of 20 requests per client_id, then one every 10 seconds), so keep attempts few.

Never retry a 400, 401, 404 or 409 unchanged. A 401 is a token problem: re-mint once and retry the request once, then stop.

Wrap every request

For one call, curl's --retry sleeps Retry-After on a 429 and backs off on a 408, 500, 502, 503 or 504, which is safe for a GET. --retry-max-time caps the total time. An SDK client retries a connection error and a 408, 409, 429 or 5xx response on every call by default, POST included. Set its retries to 0 on a write that is not safe to repeat: POST /v2/users, POST /v2/email_recipients, and POST /v2/documents unless every document has an external_id.

curl "https://api.brellium.com/v2/audits/a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d" \
  --retry 5 \
  --retry-max-time 60 \
  --header "Authorization: Bearer $BRELLIUM_ACCESS_TOKEN"

// Like curl --retry 5 --retry-max-time 60: retry 408, 429 and 500/502/503/504,
// waiting Retry-After when sent, else 1 s doubling each attempt.
const started = Date.now();
for (let attempt = 0; ; attempt++) {
  const res = await fetch(
    "https://api.brellium.com/v2/audits/a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d",
    {
      headers: { Authorization: `Bearer ${process.env.BRELLIUM_ACCESS_TOKEN}` },
    },
  );
  if (res.ok) {
    console.log(await res.json());
    break;
  }
  const waitMs = Number(res.headers.get("retry-after") ?? 2 ** attempt) * 1000;
  const transient = [408, 429, 500, 502, 503, 504].includes(res.status);
  if (!transient || attempt === 5 || Date.now() - started + waitMs > 60_000) {
    throw new Error(`HTTP ${res.status}: ${await res.text()}`);
  }
  await new Promise((resolve) => setTimeout(resolve, waitMs));
}

import BrelliumClient from "@brellium/api-client";

const client = new BrelliumClient({ oAuth2: process.env.BRELLIUM_ACCESS_TOKEN });

// The client retries 408, 409, 429 and 5xx (2 retries by default), waiting
// Retry-After when sent.
const audit = await client.audits.retrieve("a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d", {
  maxRetries: 5,
});
console.log(audit);

import os

from brellium_api_client import BrelliumClient

client = BrelliumClient(o_auth2=os.environ["BRELLIUM_ACCESS_TOKEN"])

# The client retries 408, 409, 429 and 5xx (2 retries by default), waiting
# Retry-After when sent.
audit = client.with_options(max_retries=5).audits.retrieve(
    "a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d"
)
print(audit.to_json())

The wrapper below keeps its own bucket with the API's limits (a burst of 100, then 5 per second) and takes a token from it before each request. On a 429 it sleeps Retry-After plus up to half a second of jitter. On a 5xx or a dropped connection it retries only a GET or a call marked idempotent, after a random wait (full jitter) of up to min(20 s, 0.5 s × 2^attempt). It stops after 5 attempts, and on an error response it also stops once 60 seconds have passed since the first attempt.

// Keep credentials on the server; never in a browser or mobile app.
const BASE_URL = "https://api.brellium.com";
const TOKEN = process.env.BRELLIUM_ACCESS_TOKEN;
if (!TOKEN) throw new Error("BRELLIUM_ACCESS_TOKEN is not set");

const sleep = (ms: number) =>
  new Promise<void>((resolve) => setTimeout(resolve, ms));

// A client-side token bucket matched to the server's: 100 burst, 5 per second.
// With N worker processes on one organization, give each one a share
// (for example `new TokenBucket(20, 1)` in each of five workers).
class TokenBucket {
  private tokens: number;
  private last = Date.now();
  constructor(
    private readonly capacity: number,
    private readonly refillPerSecond: number,
  ) {
    this.tokens = capacity;
  }
  async take(): Promise<void> {
    for (;;) {
      const now = Date.now();
      this.tokens = Math.min(
        this.capacity,
        this.tokens + ((now - this.last) / 1000) * this.refillPerSecond,
      );
      this.last = now;
      if (this.tokens >= 1) {
        this.tokens -= 1;
        return;
      }
      await sleep(((1 - this.tokens) / this.refillPerSecond) * 1000);
    }
  }
}

const bucket = new TokenBucket(100, 5);

export class BlockedError extends Error {}
export class RetriesExhaustedError extends Error {}
export class ApiError extends Error {
  constructor(
    readonly status: number,
    readonly code: string,
    readonly requestId: string | undefined,
    message: string,
  ) {
    super(message);
  }
}

type RequestOptions = {
  method?: "GET" | "POST" | "PATCH" | "DELETE";
  body?: unknown;
  // Set true only for calls that leave the same state when repeated,
  // such as PATCH /v2/audits/{audit_id} with the same flags.
  idempotent?: boolean;
  maxAttempts?: number;
  maxTotalMs?: number;
};

// Full jitter: a random wait between 0 and min(20 s, 0.5 s * 2^attempt).
function backoffMs(attempt: number): number {
  return Math.random() * Math.min(20_000, 500 * 2 ** attempt);
}

export async function brelliumRequest<T>(
  path: string,
  opts: RequestOptions = {},
): Promise<T> {
  const method = opts.method ?? "GET";
  const retryable = method === "GET" || opts.idempotent === true;
  const maxAttempts = opts.maxAttempts ?? 5;
  const maxTotalMs = opts.maxTotalMs ?? 60_000;
  const started = Date.now();

  for (let attempt = 0; ; attempt++) {
    await bucket.take();

    let res: Response;
    try {
      res = await fetch(`${BASE_URL}${path}`, {
        method,
        headers: {
          Authorization: `Bearer ${TOKEN}`,
          ...(opts.body === undefined
            ? {}
            : { "Content-Type": "application/json" }),
        },
        body: opts.body === undefined ? undefined : JSON.stringify(opts.body),
      });
    } catch (err) {
      // The request may or may not have reached the API: indeterminate.
      if (!retryable || attempt + 1 >= maxAttempts) throw err;
      await sleep(backoffMs(attempt));
      continue;
    }

    const isJson = (res.headers.get("content-type") ?? "").includes(
      "application/json",
    );

    if (res.ok) {
      // One token refills every 200 ms; pause that long when the bucket just hit
      // zero.
      if (res.headers.get("ratelimit-remaining") === "0") await sleep(200);
      // A DELETE answers 204 with no body.
      if (res.status === 204) return undefined as T;
      return (await res.json()) as T;
    }

    if (res.status === 403 && !isJson) {
      // A 403 with no JSON body: the request was refused (too many requests
      // from this IP address, or the query string is too long). Do not retry.
      throw new BlockedError("Bare 403: stop sending and lower concurrency");
    }

    const retryThis = res.status === 429 || (retryable && res.status >= 500);
    if (!retryThis) {
      const body = isJson
        ? ((await res.json()) as { error?: { code?: string; message?: string } })
        : {};
      throw new ApiError(
        res.status,
        body.error?.code ?? "HTTP_ERROR",
        res.headers.get("x-request-id") ?? undefined,
        body.error?.message ?? `HTTP ${res.status}`,
      );
    }

    const elapsed = Date.now() - started;
    if (attempt + 1 >= maxAttempts || elapsed >= maxTotalMs) {
      throw new RetriesExhaustedError(
        `${method} ${path} still failing with ${res.status} after ${attempt + 1} ` +
          `attempts (request id ${res.headers.get("x-request-id")})`,
      );
    }

    const retryAfter = Number(
      res.headers.get("retry-after") ?? res.headers.get("ratelimit-reset"),
    );
    const waitMs =
      res.status === 429 && Number.isFinite(retryAfter) && retryAfter > 0
        ? retryAfter * 1000 + Math.random() * 500
        : backoffMs(attempt);
    await sleep(Math.min(waitMs, maxTotalMs - elapsed));
  }
}

// Usage: the read from step 1. For a flags update, pass
// { method: "PATCH", body: { flags }, idempotent: true }.
const audit = await brelliumRequest<{ audit_id: string; flags: string[] }>(
  "/v2/audits/a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d",
);
console.log(audit.flags);

# Keep credentials on the server; never in a browser or mobile app.
import os
import random
import time

import requests

BASE_URL = "https://api.brellium.com"
TOKEN = os.environ["BRELLIUM_ACCESS_TOKEN"]


class TokenBucket:
    """A client-side bucket matched to the server's: 100 burst, 5 per second.

    With N worker processes on one organization, give each one a share
    (for example TokenBucket(20, 1) in each of five workers).
    """

    def __init__(self, capacity: float, refill_per_second: float) -> None:
        self.capacity = capacity
        self.refill = refill_per_second
        self.tokens = capacity
        self.last = time.monotonic()

    def take(self) -> None:
        while True:
            now = time.monotonic()
            self.tokens = min(
                self.capacity, self.tokens + (now - self.last) * self.refill
            )
            self.last = now
            if self.tokens >= 1:
                self.tokens -= 1
                return
            time.sleep((1 - self.tokens) / self.refill)


bucket = TokenBucket(100, 5)


class Blocked(Exception):
    pass


class RetriesExhausted(Exception):
    pass


class ApiError(Exception):
    def __init__(
        self, status: int, code: str, request_id: str | None, message: str
    ) -> None:
        super().__init__(f"{status} {code}: {message} (request id {request_id})")
        self.status = status
        self.code = code
        self.request_id = request_id


def backoff_seconds(attempt: int) -> float:
    # Full jitter: a random wait between 0 and min(20 s, 0.5 s * 2^attempt).
    return random.uniform(0, min(20.0, 0.5 * 2**attempt))


def brellium_request(
    path: str,
    method: str = "GET",
    json: dict | None = None,
    idempotent: bool = False,
    max_attempts: int = 5,
    max_total_seconds: float = 60.0,
):
    # idempotent=True only for calls that leave the same state when repeated,
    # such as PATCH /v2/audits/{audit_id} with the same flags.
    retryable = method == "GET" or idempotent
    started = time.monotonic()
    attempt = 0
    while True:
        bucket.take()
        try:
            # requests sets Content-Type: application/json when json= is given.
            res = requests.request(
                method,
                f"{BASE_URL}{path}",
                json=json,
                headers={"Authorization": f"Bearer {TOKEN}"},
                timeout=30,
            )
        except requests.RequestException:
            # The request may or may not have reached the API: indeterminate.
            if not retryable or attempt + 1 >= max_attempts:
                raise
            time.sleep(backoff_seconds(attempt))
            attempt += 1
            continue

        is_json = "application/json" in res.headers.get("content-type", "")

        if res.ok:
            # One token refills every 200 ms; pause that long when the bucket just
            # hit zero.
            if res.headers.get("ratelimit-remaining") == "0":
                time.sleep(0.2)
            # A DELETE answers 204 with no body.
            if res.status_code == 204:
                return None
            return res.json()

        if res.status_code == 403 and not is_json:
            # A 403 with no JSON body: the request was refused (too many requests
            # from this IP address, or the query string is too long). Do not retry.
            raise Blocked("Bare 403: stop sending and lower concurrency")

        retry_this = res.status_code == 429 or (
            retryable and res.status_code >= 500
        )
        if not retry_this:
            error = res.json().get("error", {}) if is_json else {}
            raise ApiError(
                res.status_code,
                error.get("code", "HTTP_ERROR"),
                res.headers.get("x-request-id"),
                error.get("message", f"HTTP {res.status_code}"),
            )

        elapsed = time.monotonic() - started
        if attempt + 1 >= max_attempts or elapsed >= max_total_seconds:
            raise RetriesExhausted(
                f"{method} {path} still failing with {res.status_code} after "
                f"{attempt + 1} attempts "
                f"(request id {res.headers.get('x-request-id')})"
            )

        retry_after = res.headers.get("retry-after") or res.headers.get(
            "ratelimit-reset"
        )
        if res.status_code == 429 and retry_after and float(retry_after) > 0:
            wait = float(retry_after) + random.uniform(0, 0.5)
        else:
            wait = backoff_seconds(attempt)
        time.sleep(min(wait, max_total_seconds - elapsed))
        attempt += 1


if __name__ == "__main__":
    # Usage: the read from step 1. For a flags update, pass
    # method="PATCH", json={"flags": [...]}, idempotent=True.
    audit = brellium_request("/v2/audits/a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d")
    print(audit["flags"])

When the API refuses a request with a 429, the wrapper waits about a second (Retry-After plus jitter) and sends it again. The 429 has these headers and this body:

HTTP/2 429
content-type: application/json
retry-after: 1
ratelimit-limit: 100
ratelimit-remaining: 0
ratelimit-reset: 1
x-request-id: 9c3f1a7e-5b2d-4e8f-a1c6-2d7b8e9f0a1b

{
  "error": {
    "type": "rate_limit_error",
    // branch on this, never on message
    "code": "RATE_LIMITED",
    "message": "Too many requests",
    // send this to Brellium support, never a response body
    "request_id": "9c3f1a7e-5b2d-4e8f-a1c6-2d7b8e9f0a1b"
  }
}

Retry-After is whole seconds and equals RateLimit-Reset: the time until one token refills, rounded up. request_id equals the x-request-id header; log it with the status on every retry. POST /v2/oauth/token refuses the 21st token request in a burst with the same headers and code.

Send Content-Type: application/json with every /v2 request body; any other content type, or none, returns 400 INVALID_CONTENT_TYPE and changes nothing.

Tell a bare 403 from an API error

No /v2 endpoint returns 403. The only 403 with a JSON body is access_denied from POST /v2/oauth/token. Any other 403 is bare, with no JSON envelope, no x-request-id and no Retry-After; the API's 429 has all three.

A 403 with no JSON body means the request was refused before processing: too many requests from one IP address in a short time, a query string over 2,048 bytes, or a request refused for security reasons.

Check the request, then pause and lower concurrency. Keep your access token and stop retrying, because a retry loop keeps the block in place.

A bare 413, 414 or 431 means the body, the URL or the headers are too large, so shorten them. For every limit, see Rate limits and errors.

Common mistakes

Mistake What you see Fix
A tight retry loop on 429 A stream of 429s with Retry-After: 1; every worker starves Sleep Retry-After plus jitter, cap attempts at 5 and total time at 60 seconds, then raise an error.
Minting a token per request 429 RATE_LIMITED from POST /v2/oauth/token after the 20th token request in a burst Mint once, cache until expires_in, re-mint only on 401 (Mint one access token and reuse it).
Many workers, one organization, no limiter in your code RateLimit-Remaining collapses to 0 every few seconds; workers take turns being refused The bucket is per organization, not per key: share one bucket, or split 5 per second across workers.

Next

Mint one access token and reuse it

Mint a token once, cache it until expires_in, re-mint on 401, wait on 429.

Mint one access token and reuse it

Rate limits and errors

The rate limits and their headers, how to retry a 429, and every error code.

Rate limits and errors

Keep audits in sync with updated_after

Backfill once, then poll updated_after windows that overlap and upsert by id.

Keep audits in sync with updated_after
Previous
Test end to end with sandbox credentials
Next
Generate a typed client from the OpenAPI document