Skip to main content
Use prompt caching when the same large context appears across many requests. Mark stable instructions, documents, examples, tool outputs, or tool definitions as cacheable so supported providers can reuse that context on later calls. Prompt caching is different from response caching. Prompt caching still runs inference, but can reduce repeated input processing cost and latency. Response caching returns a previously generated answer for an identical request.
Prompt caching is provider- and model-specific. Phaseo sends each provider only the cache controls it documents and removes the rest, and cache controls never exclude a provider from routing. If a provider reports cached tokens on a route that has no cache price yet, Phaseo bills cache reads at the route’s normal input rate and cache writes at the input rate times the provider’s documented write multiplier. Check the model page pricing table for cache read/write rates.

​
What to cache

Cache content that stays stable across requests:
  • long system instructions
  • reused RAG documents
  • few-shot examples
  • tool definitions
  • large tool results that are reused in the next turn
Avoid caching content that changes every request, contains short one-off user input, or includes sensitive data that your policy does not allow to be stored by the selected provider.

​
Automatic and opt-in caching

Most providers cache repeated prompt prefixes automatically. Claude needs an explicit opt-in. Other OpenAI-compatible providers receive prompt_cache_key when their API accepts it, and no other cache controls. Phaseo always sends a hash of your workspace and key rather than the key itself. See Provider affinity.

​
Cache controls

​
Default policy for Claude

Phaseo accepts a top-level cache_control on Chat Completions, Responses, and Anthropic Messages requests:
It sets a default cache policy for Claude routes: Phaseo places the cache breakpoints for you. Use ttl: "5m" (the default) for short-lived shared context and ttl: "1h" when the model supports longer-lived prompt cache entries. Other providers ignore it. A top-level cache_control also keeps the conversation on one provider; see Provider affinity. provider_options.anthropic.cache_control is the same setting. When a request sends both, the provider_options value wins:
Supported scope values: Per-block cache_control wins over the default policy. Anthropic accepts at most four cache breakpoints per request. Default policies count the cache_control blocks you set yourself and stop at four. Unrecognised scope values are treated as agent. Use agent for multi-step tool loops: all_text and last_user_message don’t mark messages that contain only tool results, so the cached prefix stops growing. If your own blocks carry more than four marks, Phaseo keeps the first and the latest three and removes the others, so the request is not rejected.

​
Cache breakpoints on blocks

Place a cache mark on a content block to cache everything up to and including that block. Phaseo accepts two spellings:
  • cache_control: {"type": "ephemeral"}, with an optional ttl of 5m or 1h.
  • prompt_cache_breakpoint: OpenAI’s {"mode": "explicit"}, with an optional ttl of 30m or 12h (see 12-hour cache on OpenAI). When a block carries both, prompt_cache_breakpoint wins.
On Chat Completions and Responses, marks are accepted on text and image parts, on function tool definitions, and on tool results whose content is an array of text parts. Anthropic Messages accepts cache_control wherever Anthropic does. Each provider receives the mark in its own form: Marks on tool definitions reach only Claude routes, Alibaba Model Studio, Venice, and Inference.net’s Claude and Gemini 2.5 models; OpenAI does not accept breakpoints on tools. Removing a mark leaves the rest of the prompt unchanged, so the cached prefix matches an unmarked request. Amazon Bedrock and Google Vertex AI cache Claude 3.7 Sonnet, 3.5 Sonnet, and 3 Opus for 5 minutes only, and Bedrock can reject a ttl on those models. On those routes Phaseo sends your breakpoints without ttl, so they cache for 5 minutes, and reports a removed 1h as cache_control.ttl in the X-Phaseo-Dropped-Params header and meta.routing.dropped_params. Inference.net documents breakpoints without a ttl, so Phaseo removes and reports a ttl there in the same way, and the writes are billed as 5-minute writes.

​
OpenAI retention and options

OpenAI’s cache controls depend on the model generation. Phaseo sends each one only where OpenAI accepts it: On GPT-5.6 and later, OpenAI accepts a ttl of 30m (the default) or 12h, and Azure OpenAI only 30m. explicit mode caches only the breakpoints you place. DeepInfra uses the same object with a ttl of 5m or 1h, and needs prompt_cache_key on every request in the session. Moonshot AI takes mode: "implicit" and a ttl of 5m (the default) or 1h. Phaseo sends 1h only to Kimi K3 routes that have a 1-hour write price, so those writes are billed at that price; elsewhere it removes the ttl and reports it as prompt_cache_options.ttl in the X-Phaseo-Dropped-Params header. Other providers receive neither field. Send either field at the top level or under provider_options.openai. When both are present, the top-level value wins:

​
12-hour cache on OpenAI

On GPT-5.6 and later, OpenAI keeps a cache entry for at least 12 hours when you set ttl: "12h", either for the whole request in prompt_cache_options or for one breakpoint in prompt_cache_breakpoint. A breakpoint without its own ttl uses the request’s. OpenAI bills a 12-hour write at 2.5× the input rate and a 30-minute write at 1.25×; cache reads cost the same for both.
  • OpenAI documents the 12-hour lifetime on its Responses API only. A Chat Completions request that asks for it runs on OpenAI’s Responses API, and the response keeps the Chat Completions format.
  • Routes that take a 12-hour cache (OpenAI’s global API) are tried first. Any other route, such as Azure OpenAI, another provider or a model before GPT-5.6, gets the request without it, so the 30-minute default applies. The request still runs, and the removed field is reported as prompt_cache_options.ttl or prompt_cache_breakpoint.ttl in the X-Phaseo-Dropped-Params header and meta.routing.dropped_params.

​
Google Gemini

Gemini caches implicitly on Gemini 2.5 and later, with nothing to send. To use an explicit cache you created with Google’s caching API, pass its name in provider_options.google.cached_content:
  • On Google AI Studio, cached_content works only with your own Google API key (BYOK). With Phaseo’s key the request returns 400, because the cache belongs to the account that created it.
  • On Vertex AI, cached_content works only with your own Vertex AI credential; with Phaseo’s credential the request returns 400. Send the cache’s full name, projects/{project}/locations/{location}/cachedContents/{id}, and Phaseo sends the request to that project and location. A short name such as cachedContents/abc123 uses the project_id and region in your service-account JSON, and returns 400 if either is missing.
  • Google does not support explicit caches on Gemini 4 Argon and later.
provider_options.google.cache_control and provider_options.google.cache_ttl are accepted but not sent to Google.

​
Chat Completions

Use /v1/chat/completions when you are using OpenAI-compatible chat clients.

​
Responses

Use /v1/responses for new OpenAI-compatible text integrations and agent flows.
Phaseo does not store responses. Requests that set previous_response_id or conversation return 400 unsupported_param, so send the whole conversation in input on every step. Its stable opening is the prefix that gets cached. In tool loops, send reasoning output items back too; see Reasoning in tool loops.

​
Anthropic Messages

Use /v1/messages when your client is Anthropic-compatible.
Anthropic Messages supports cache control on:
  • system text blocks
  • message text and image blocks
  • tool result blocks
  • tool definitions
When several TTLs appear in one request, Anthropic requires the 1h breakpoints to come before the 5m ones.

​
Usage and pricing fields

Usage reports cache reads and writes on every text endpoint. Chat Completions and Responses follow OpenAI’s semantics: the input count includes cache reads and cache writes. Anthropic Messages follows Anthropic’s: input_tokens excludes them. On Chat Completions and Responses, uncached input is the total input minus cache reads minus cache writes. Streamed Anthropic Messages responses carry the cache fields on message_delta. Chat Completions usage has no split by lifetime, because OpenAI documents none there; the request details still record it.
Earlier responses reported cache writes as output_tokens_details.cached_tokens (completion_tokens_details.cached_tokens on Chat Completions), and Claude routes left cache reads and writes out of input_tokens on Chat Completions and Responses. Neither applies any more: read writes from cache_write_tokens. Non-streamed Anthropic Messages responses used to return cache_creation_input_tokens: null; they now return the real counts.
Request details and pricing breakdowns use these meters:

​
How cached tokens are priced

Phaseo bills each input token once, at exactly one rate:
  • uncached input at the input rate
  • cache reads at the cache-read rate
  • cache writes at the cache-write rate, or at the 5-minute and 1-hour rates (Claude) or the 30-minute and 12-hour rates (OpenAI) when the provider prices them separately
Some providers count cache writes inside the input total and others report them on top of it. Phaseo bills both the same way, so a written token is never also billed as ordinary input. For example, a GPT-5.6 Sol request with 53,100 input tokens, of which 48,000 are cache reads and 3,000 are cache writes, plus 850 output tokens, costs: If 2,000 of those writes had a 12-hour lifetime, they would cost $12.50 per 1M instead of $6.25, and the request would cost $0.09125. If a provider reports cached tokens on a route that has no cache price in the catalogue yet, Phaseo does not fail the request:
  • Cache reads are billed at the route’s input rate.
  • Cache writes are billed at the input rate times the provider’s documented write multiplier: 1.25× for Claude 5-minute writes and 2× for 1-hour writes; 1.25× for OpenAI, Azure OpenAI, and Alibaba; 1× for Kimi K3 5-minute writes and 2× for 1-hour writes; 2.5× for OpenAI 12-hour writes on any route; 1× for other providers.
  • The pricing line is marked pricing_fallback: "input_rate".
On a Claude route whose catalogue has one cache-write price, that price is the 5-minute rate. Writes the provider reports as 1-hour writes are billed at 2× the input rate, on a cached_write_text_tokens_1h line marked pricing_fallback: "input_rate". The final message_delta of a streamed Claude response reports the cumulative cache-write total without the TTL split. Writes beyond the split are billed as 5-minute writes, because server-tool writes can use 5 minutes even when the request asks for 1 hour. Some providers do not discount cached tokens on some models. Their model pages show the same price for input and cache reads.

​
Practical checks

After you add prompt caching:
  1. Send one request to create or warm the cache.
  2. Send a second request with the same cacheable content.
  3. Check cached_tokens and cache_write_tokens in the response usage, and the cache meters in the request details.
  4. Compare latency and cost over repeated calls, not just the first call.

​
Provider affinity

A prompt cache lives with one provider, and often with one region or deployment of that provider. Phaseo keeps related requests on the provider and route that served them, so an agent loop does not pay to rebuild its cache on a different provider. Send prompt_cache_key or session_id, or opt into caching with cache_control (top-level, provider_options.anthropic.cache_control, or a cache_control or prompt_cache_breakpoint mark on a message block or tool), to pin a conversation. Phaseo pins it to the provider and route that served the first successful request, and later requests with the same key go there too. Pins are kept per workspace. How long a pin lasts depends on the provider’s cache retention: A pin moves only when its provider fails (an open circuit breaker or a high recent error rate), when Phaseo’s managed quota on that provider is used up, or when the active route, preset, guardrail, or request policy no longer allows it. The request then fails over, and the pin moves to the provider that served it. Request details record this as sticky_routing.demotionReason in the routing diagnostics. Pinned requests are not used to probe recovering providers. Without a key, session_id or cache mark, Phaseo still pins the opening context for 15 minutes to a provider that reported a cache read or a cache write for it, as long as that provider’s cached input is cheaper than its standard input. Phaseo sends a hash of your workspace and prompt_cache_key to providers, never the raw key, so the same key in two workspaces never shares a provider cache. With only a session_id, Phaseo derives that hash from the session for OpenAI, xAI, Fireworks and Cerebras. Fireworks, Cloudflare Workers AI and Baseten also receive it in the x-session-affinity header, Morph in the x-session-id header, and xAI receives it as x-grok-conv-id unless you set provider_options.xai.conversation_id. Responses still echo your own prompt_cache_key. Set provider.cache_aware_routing to false to opt out for one request. Set routing.session_affinity to false when the request carries a session_id but should use only normal context-based routing. Phaseo then also stops deriving a provider cache key from the session.
Last modified on October 21, 2026