Skip to main content
A request can be throttled by the selected provider or by an applicable workspace or key policy. Routing and fallbacks can reduce provider failures, but a request can still return 429.

​
Gateway limits

Each user in each workspace has a requests-per-minute allowance, measured over a rolling minute. API keys created by the same user share that allowance. OAuth requests use the authenticated user. Keys without an owner share a workspace allowance. The allowance depends on the workspace’s account standing and rises automatically as the workspace becomes established: A workspace receives the highest allowance it qualifies for. Changes take effect within a few minutes. Free-model routes also have a default allowance of 1500 requests per UTC day, shared across free models for the same user and workspace. Paid-model requests do not consume this daily allowance. An admitted free request counts even if the provider subsequently fails; provider fallbacks within that request count once. These limits apply across gateway regions. They can be increased for a specific user and workspace. File operations and job-status polling use the minute allowance; management endpoints do not. A gateway quota rejection returns 429 with Retry-After in seconds. Its error reason is customer_requests_per_minute or free_requests_per_day. Wait for that delay before retrying. If quota enforcement is temporarily unavailable, the gateway returns 503 with retry guidance.

​
Key limits and budgets

An API key’s daily, weekly or monthly request and spend limits, and the workspace’s budgets, return 429 with the key_limit_exceeded error code once reached. The body’s reason, limit_window and reset_at fields identify the limit and when its window resets. Retry-After gives the seconds until that reset. It is omitted when the limit does not reset by itself, such as a lifetime budget or a soft-blocked key; instead of retrying, raise the workspace budget, or for a key limit, raise the key’s limit or use a different key. Workspace owners and admins can also set, change and delete budgets with phaseo budgets create|update|delete or in the dashboard under Settings → Budgets. The page shows each budget’s spend so far, the share used, a projection for the period and when it resets. Daily, weekly and monthly budgets reset on UTC boundaries: midnight, Monday 00:00 and the 1st of the month. Department budgets are managed through the /v1/department-budgets API. A department’s spend counts the same keys and OAuth requests as department rate limits, below. Once a department has used its daily, weekly or monthly budget, API-key requests that count toward it return 429 with the key_limit_exceeded error code and reason department_daily_cost_budget_reached, department_weekly_cost_budget_reached or department_monthly_cost_budget_reached. The body’s department_id and department_name fields identify the department, and reset_at and Retry-After say when its budget resets. Batch jobs, realtime sessions and OAuth requests are not blocked by department budgets. If the budget check is briefly unavailable, requests are allowed rather than refused, so a department can spend past its budget for a short time. Owners and admins get a mobile push at about 80% and again at 100% of a budget. These alerts come from usage that can lag by up to an hour, so requests can be refused before the 100% alert arrives. Owners and admins can also manage department budgets with phaseo department-budgets, under Settings → Budgets, or with the Terraform provider’s phaseo_department_budget resource.

​
Rate limit policies

Workspace admins can create named rate limit policies through the /v1/rate-limits management API and assign them to API keys, workspace members, departments, guardrails, or as the default for each end user. The gateway enforces a policy’s requests_per_minute and tokens_per_minute, each measured over a rolling minute:
  • Key: the key’s own allowance.
  • Member: one allowance shared by the member’s keys and OAuth requests.
  • Department: one allowance shared by everything that counts toward the department. A key linked to a department counts toward that department only. Any other key counts toward every department its creator belongs to, and OAuth requests count toward every department of their user. A member with a department override counts toward the override department only.
  • Guardrail: a separate allowance for each key (or OAuth user) the guardrail covers.
  • End-user default: a separate allowance for each end user, identified by the request’s user field or X-Phaseo-User-Id header. Requests without an end-user ID are not limited by it.
Owners and admins can also manage policies and their assignments with phaseo rate-limits policies and phaseo rate-limits assignments, under Settings → Rate limits, or with the Terraform provider’s phaseo_rate_limit_policy and phaseo_rate_limit_assignment resources. When several policies apply, each is counted and the first one exhausted rejects the request. Policy limits apply in addition to the gateway limits above. Policy changes take effect within about a minute. Department membership changes made in the dashboard or through SCIM can take up to about five minutes. tokens_per_minute counts each request’s input and output tokens once it completes, including streamed and failed requests that used tokens. A request is rejected once an allowance’s usage over the last minute reaches its limit, so requests already in progress can take usage past the limit. A rejected request returns 429 with the rate_limit_exceeded error code, reason policy_requests_per_minute or policy_tokens_per_minute, the policy_id and the limit_scope (key, member, department, guardrail or end_user), plus Retry-After in seconds. Responses to requests a policy applies to include x-ratelimit-limit-requests, x-ratelimit-remaining-requests and x-ratelimit-reset-requests (for example 60s) for the most restrictive request limit, and x-ratelimit-limit-tokens, x-ratelimit-remaining-tokens and x-ratelimit-reset-tokens for the most restrictive token limit. A rejected request carries only the headers of the limit that rejected it. Counting happens in the background, so remaining is approximate, and token headers do not yet include the current request’s own usage. Provider fallbacks within one request count once, and response cache hits are not counted. concurrent_requests limits how many requests can be in progress at once. A request holds a slot from when it is admitted until its response has been sent in full, including the whole of a streamed response, and provider and model fallbacks within the request share that slot. When every slot is in use, the request is rejected with reason policy_concurrent_requests and Retry-After: 1. Checking a concurrency limit waits for the policy’s counter, which adds a little latency to the requests it covers. A slot whose release is lost frees itself after 15 minutes. Batches and realtime sessions are not limited by it, and a video generation holds a slot only while the request that starts it is in progress. By default a policy fails open: if its limits cannot be checked, the request is allowed. Set fail_closed to make it strict. A strict policy’s request and token limits are checked before the request is sent rather than in the background, which makes them exact but adds latency, and if they cannot be checked within two seconds the request is rejected with reason policy_unavailable. Concurrency and policy_unavailable rejections carry no x-ratelimit-* headers.

​
Provider limits

BYOK requests use the limits of your provider account. Managed requests use shared provider capacity; that capacity is not an individual workspace allowance. Phaseo forwards a validated Retry-After value when the routed provider supplies one. BYOK responses can also include X-Phaseo-Upstream-RateLimit-* headers. These quota headers are omitted for managed requests because they describe shared provider capacity. Other upstream headers are not forwarded.

​
Handle a throttled request

  1. Inspect the error code to distinguish temporary provider throttling from a policy or budget restriction that requires configuration changes.
  2. Respect Retry-After when present. It can be a delay in seconds or an HTTP date; validate the value before scheduling a retry.
  3. Otherwise, use exponential backoff with jitter and a maximum delay.
  4. Bound retries by both attempt count and an overall deadline. Cancel abandoned work and release each failed response body before retrying.
  5. Reduce concurrency if throttling persists.
Do not automatically retry an ambiguous job submission or replay completed tool actions. A request may have reached the provider even if the client lost its response; retries can duplicate work and charges. For an accepted video or batch job, use its status endpoint rather than submitting it again.

​
Monitor throttling

Track 429 rates by endpoint and model, and inspect fallback frequency in request logs. Compare your application logs with routing settings and provider account quotas for BYOK traffic. See budgets and guardrails for workspace spending and access controls.
Last modified on October 29, 2026