Handle LLM API Rate Limits: 429, Retry-After, and Free-Tier Quota Math (2026)

Direct answer

Rate-limit handling has five rules that cover almost every free-tier failure mode:

1. Treat 429 as three different errors. Rapid-ramp throttling (slow_down), hard rate limits, and exhausted quota/billing can all arrive as HTTP 429. Only the first two are retryable — OpenAI's official guide says explicitly: don't retry quota or billing errors. OpenRouter returns 402 when your balance is negative, even on :free models.
2. Retry-After is a minimum, not the exact wait. OpenAI's guide: wait at least that long, then add a small random delay. The pattern below is full jitter on top of the floor: sleep = (Retry-After ?? 0) + rand(0, min(cap, base × 2^(attempt−1))), with attempt starting at 1 so the first retry's ceiling is base.
3. Failed attempts still count. OpenAI: "unsuccessful requests contribute to your per-minute limit." OpenRouter (official help center): "Failed attempts still count toward your daily quota." Hammering a 429 makes the hole deeper.
4. Compute the binding budget, not the headline numbers. Effective daily requests = min(RPM × 1440, RPD). On most free tiers the daily cap binds at under 5% of the per-minute capacity — pacing at the daily rate is what keeps you unstopped.
5. Read the headers, and check what they actually mean. On Groq, x-ratelimit-limit-requests is requests per day and x-ratelimit-limit-tokens is tokens per minute — RPM is never exposed. Misreading those headers is why people "pause and still get 429s."

Everything below is sourced from provider documentation accessed 2026-10-12, plus our verified free-tier dataset (18 rows, last verified 2026-10-12) on the free LLM API limits page.

Why 429 does not always mean "slow down"

Developers treat 429 as one error. Providers officially document at least four distinct causes:

One caveat on the 503 row: it is OpenAI's server_is_overloaded. OpenRouter's 503 means no eligible provider for your routing configuration — relax provider preferences or add fallback models instead of burning retries.

Two details that trip up real projects:

• Ramp-rate throttling is invisible in usage dashboards. OpenAI documents that slow_down "can occur even when your traffic is within its requests-per-minute and tokens-per-minute limits" — it reflects how fast traffic increased, not whether you exhausted a quota. Their rule of thumb above 1M input TPM: increase by no more than 50% every 15 minutes.
• The same 429 text hides different semantics across providers. A widely-followed OpenAI community thread shows dashboards far under the documented limits while receiving 429 floods — the thread's resolution: OpenAI historically used 429 for rate limits, monthly quota, and overload. On GitHub's meta-llama repo, users paused jobs for minutes and got instant 429s again — the confusion traced back to which window (requests vs tokens) was actually exhausted.

This is also why the pain shows up everywhere — from Claude Code users hitting usage limits 'way faster than expected' to open-source schedulers that suspend agents before the limit kills them. The community threads track the same failure mode: r/ChatGPTCoding on Claude Pro subscription limits and r/LocalLLaMA's pre-emptive scheduler discussion. Same root problem: quotas are multi-dimensional, and clients that only sleep-and-retry lose.

What the headers actually tell you

OpenAI and Groq publish the same header family (values illustrative, from official docs):

Cerebras adds two twists worth knowing before you build on it: limits are enforced as two token buckets (uncached TPM and total TPM — cached tokens don't count against the uncached bucket), and quotas are checked at both project and organization level (Cerebras rate limits).

Free-tier quota math (worked examples)

The headline "30 RPM" on a free tier is almost never the number that stops you. The binding daily ceiling is:

effective_requests_per_day = min(RPM × 1440, RPD)

If the tier also has a tokens-per-day cap, that is a third term — a request-count ceiling only holds while your average request stays small enough not to trip the token cap first:

effective_requests_per_day = min(RPM × 1440, RPD, tokens_per_day ÷ tokens_per_request)

The first two terms are published, provider-enforced limits. The third is derived from your traffic mix, which is why the same tier can be generous for one workload and fatal for another (worked example: Cerebras below).

Using the dataset figures on our free-tier comparison (rows verified 2026-09-26 – 2026-10-12) and OpenRouter's official limits (checked 2026-10-12):

(Calculations verified 2026-10-12. OpenRouter's $10 tier jump is all-time credits purchased, not current balance — and the docs' rounding note grants the 1,000/day ceiling starting one credit below the threshold (currently 9 credits) "to absorb rounding and top-up fees". Cerebras publishes no requests-per-day counter: 7,200/day (5 RPM × 1,440) is only reachable for requests under ~139 tokens each, so the 1M tokens/day cap binds first for any realistic response size — hence the Cerebras ceiling above is stated at 2K tokens/resp, matching the token table below.)

Token budgets matter just as much. A 1M tokens/day budget (Cerebras Free Trial-style) yields:

So "1M tokens/day" sounds generous until your agent emits 8K-token answers: you get 125 turns/day, or one every ~11.5 minutes. Two levers, both official guidance: set max_tokens close to your real output size (OpenAI computes your rate-limit consumption as the max of max_tokens and the estimated prompt tokens), and batch multiple tasks into one request when you are RPM-bound but TPM-light.

A Retry-After-aware backoff client (dependency-free, tested)

The client below implements the two official rules: Retry-After as a floor, full jitter on top, attempt and wall-clock caps, and no retries on non-retryable errors. If you use an official SDK, disable its internal retries (or account for them) so you don't get nested retry loops multiplying traffic — OpenAI's guide calls this out explicitly.

// Verified with Node.js on 2026-10-12. Seeded rng makes outputs reproducible.

type RetryableError = {
  status: number;                 // HTTP status
  code?: string;                  // e.g. "insufficient_quota", "slow_down"
  headers?: Record<string, string>;
};

// Documented OpenAI 429 codes that need a human, not a retry.
const NON_RETRYABLE_429_CODES = new Set([
  "insufficient_quota",                // out of credits
  "credit_balance_exhausted",          // no prepaid credits left
  "organization_spend_limit_exceeded", // org hard spend limit
  "project_spend_limit_exceeded",      // project hard spend limit
]);

export function isRetryable(err: RetryableError): boolean {
  if (err.status === 503) return true;                    // overload — retry
  if (err.status !== 429) return false;
  if (err.code && NON_RETRYABLE_429_CODES.has(err.code)) return false; // billing/spend — do not retry
  return true;                                            // ramp or hard rate limit
}

export function parseRetryAfter(headers: Record<string, string> | undefined, now = Date.now()): number | null {
  // header names are case-insensitive (RFC 9110), so accept either casing
  const v = headers?.["retry-after"] ?? headers?.["Retry-After"];
  if (!v) return null;
  const secs = Number(v);
  if (Number.isFinite(secs) && secs >= 0) return secs;     // Groq/OpenAI send seconds
  const asDate = Date.parse(v);                            // RFC 1123 form
  return Number.isNaN(asDate) ? null : Math.max(0, (asDate - now) / 1000);
}

export function fullJitterDelay(attempt: number, base = 1, cap = 60, rng: () => number = Math.random): number {
  return rng() * Math.min(cap, base * 2 ** (attempt - 1)); // AWS "full jitter"
}

export async function withRetry<T>(
  fn: () => Promise<T>,
  opts: {
    maxAttempts?: number; maxElapsedSec?: number; base?: number;
    cap?: number; sleep?: (s: number) => Promise<void>;
  } = {}
): Promise<T> {
  const {
    maxAttempts = 6, maxElapsedSec = 120, base = 1, cap = 60,
    sleep = (s) => new Promise((r) => setTimeout(r, s * 1000)),
  } = opts;
  const startedAt = Date.now();
  let lastErr: unknown;
  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    try {
      return await fn();
    } catch (err) {
      lastErr = err;
      if (!isRetryable(err as RetryableError)) throw err;
      const floor = parseRetryAfter((err as RetryableError).headers) ?? 0;
      const delay = floor + fullJitterDelay(attempt, base, cap);
      if ((Date.now() - startedAt) / 1000 + delay > maxElapsedSec) throw err;
      if (attempt === maxAttempts) throw err;
      await sleep(delay);
    }
  }
  throw lastErr;
}

Seeded outputs (base=1s, cap=60s, rng = mulberry32(42); the tiny PRNG below makes these numbers reproducible):

// mulberry32 — deterministic PRNG so backoff sequences are reproducible in tests
export function mulberry32(seed: number): () => number {
  let a = seed >>> 0;
  return () => {
    a = (a + 0x6d2b79f5) >>> 0;
    let t = a;
    t = Math.imul(t ^ (t >>> 15), t | 1);
    t ^= t + Math.imul(t ^ (t >>> 7), t | 61);
    return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
  };
}

attempt 1: ceiling=1s  delay=0.60s
attempt 2: ceiling=2s  delay=0.90s
attempt 3: ceiling=4s  delay=3.41s
attempt 4: ceiling=8s  delay=5.36s
attempt 5: ceiling=16s delay=2.80s
attempt 6: ceiling=32s delay=16.85s

Re-seeding mulberry32(42) with Retry-After: 7 present, the 7 s floor is still the larger term in each of the first three attempts: 7.60s, 7.90s, 10.41s.

The pacing difference is not theoretical — simulating 60 requests against a 30 RPM / 1,000 RPD budget (Groq Free Plan):

naive burst: 60 req in ~3s = 1200 RPM attempted vs 30 RPM cap
             -> first 30 accepted, 30 rejected with 429
paced:       60 req at 86.4s spacing -> ~86 min total, 0.7 RPM effective
             -> all accepted, well under both RPM and RPD

Routing patterns that survive free tiers

1. Pace to the binding limit, not the headline. If RPD binds (most free tiers), spread requests at 86400 / RPD seconds. A 50/day OpenRouter :free key is a 28.8-minute metronome, not an API you can loop.

2. Chain fallbacks by independent quotas. Quotas are per-provider (and on Cerebras, per project and org). A fallback chain — e.g. Groq → SambaNova → GitHub Models → an OpenRouter :free model — multiplies daily capacity because the budgets don't share windows (Cerebras also works here but its Free Trial requires a payment method). Current per-provider limits live in the dataset-backed comparison; endpoint and auth details for wiring the chain are in the free API directory.

3. Track UTC-day counters yourself. OpenRouter's free-model quota is a UTC-day counter, queryable via GET /api/v1/key → free_model_daily_requests (used / limit / remaining). Critically, rejected requests still increment it — so client-side admission control (skip the call if remaining === 0) beats finding out via 429.

4. Use prompt caching as rate-limit relief. Groq: "Cached tokens do not count towards your rate limits." Cerebras enforces separate uncached/total buckets (Free Trial: 30K uncached / 90K total TPM). Their docs' worked example uses a 1M uncached limit, i.e. 3M total: you need a ≥67% cache hit rate before the total bucket saturates (the docs' example uses 70%). Stable system prompts and repeated tool schemas are free quota.

Troubleshooting: 429s that shouldn't exist

When to stop fighting free tiers

Free tiers are the right answer for prototypes, CI jobs, and background features — and the wrong answer for anything with a latency SLO. Do the volume math first: if your product needs 500 two-K-token responses per day, one 1M-token budget is already fully spoken for; two consumers will 429 regardless of how good your backoff is. At that point compare API vs self-hosting breakeven, size the hardware with the VRAM calculator, and check current cheapest cloud GPU rates — or keep a paid tier for the hot path and demote free tiers to fallback, which is exactly what the routing patterns above are for.

Sources (accessed 2026-10-12 – 2026-10-13)

• OpenAI — Rate limits guide (429/503 semantics, headers, backoff, ramp-rate rule, "unsuccessful requests contribute")
• OpenAI — Spend limits and 429 troubleshooting (organization_spend_limit_exceeded, project_spend_limit_exceeded, credit_balance_exhausted — 429s that need a human, not a retry)
• Groq — Rate limits (headers, retry-after scope, cached-token exemption, free-plan table)
• Cerebras — Rate limits and Pricing (uncached/total buckets, org/project quotas, per-model free limits)
• OpenRouter — API limits and Help center: rate limits (:free tiers, 402 semantics, failed-attempts-count rule)
• OpenAI Community — Dreaded 429 rate limit errors when usage is well under the limits
• GitHub — meta-llama/llama-models#354
• BBC News — Claude Code users hitting usage limits 'way faster than expected'
• GitHub — Champoleello/agentpause: rate-limit-aware agent scheduler
• Community context (Reddit) — r/LocalLLaMA: rate-limit-aware agent scheduler, r/ChatGPTCoding: Claude Pro limits
• OpenGPU Radar — Free LLM API limits dataset, verification methodology