Parameters - Phaseo Docs

This page is the field-by-field reference for request parameters exposed by Phaseo. Use it when you want to know:

what a parameter does
what type it expects
the typical range or accepted values
whether it changes quality, cost, latency, or routing behavior

If you want tuning advice instead of field definitions, use Inference Parameters and Sampling and Decoding. Parameter support still varies by endpoint, model, and provider. The model quickstart table shows support aggregated across currently active providers for a specific route.

Unsupported parameters

If the provider serving a request does not support one of its sampling parameters, Phaseo drops that parameter. It is not sent upstream, and the request is not rejected, so clients that always send OpenAI defaults keep working. Support is decided per provider from Phaseo’s provider profiles and the route’s catalogue parameters. If a route declares no parameters, only the catalogue check is skipped: parameters the provider profile marks unsupported are still dropped, for example top_k on Cohere or stop on Reflection. Only these parameters can be dropped: temperature, top_p, top_k, min_p, repetition_penalty, frequency_penalty, presence_penalty, logit_bias, logprobs, top_logprobs, seed, stop (stop_sequences on /v1/messages), parallel_tool_calls, max_tool_calls and image_config. Messages, tools, tool_choice, response_format, reasoning settings and output limits are never dropped. Separately, a prompt-cache ttl of 12h is removed on routes that cannot take OpenAI’s 12-hour cache, and reported in the same places as prompt_cache_options.ttl or prompt_cache_breakpoint.ttl; see Prompt Caching. On successful responses, Phaseo reports the dropped parameters that the request itself sent:

in the X-Phaseo-Dropped-Params response header, for example X-Phaseo-Dropped-Params: top_k, min_p
in meta.routing.dropped_params when the request sets meta: true, for example [{"param": "top_k", "reason": "unsupported_by_provider"}]

A parameter that comes from a preset or a default value, not from the request body, can be dropped without appearing in either place. It is still recorded in the routing diagnostics that Phaseo stores with the request.

Quick lookup

Parameter	Type	Use it for
`model`	`string`	Selecting the gateway model id to run.
`models`	`string[]`	Listing ordered fallback models tried after `model` on retryable failures.
`stream`	`boolean`	Returning SSE output incrementally instead of one final payload.
`temperature`	`number`	Increasing or reducing randomness.
`top_p`	`number`	Narrowing or widening the nucleus sampling pool.
`top_k`	`integer`	Restricting sampling to the top-k candidate tokens.
`max_tokens`	`integer`	Capping output length on routes that still use this field name.
`max_output_tokens`	`integer`	Capping output length on routes that use the newer field name.
`max_completion_tokens`	`integer`	Capping output length on newer OpenAI-style text APIs.
`frequency_penalty`	`number`	Discouraging repeated tokens and phrases.
`presence_penalty`	`number`	Encouraging topic or vocabulary change.
`repetition_penalty`	`number`	Provider-specific anti-repetition control.
`seed`	`integer`	Improving reproducibility when the upstream provider supports it.
`stop`	`string` or `string[]`	Defining explicit stop sequences.
`logprobs` / `top_logprobs`	`boolean` / `integer`	Requesting token probability data.
`tools`, `tool_choice`	`array`, `string`, `object`	Tool calling and function execution control.
`parallel_tool_calls`	`boolean`	Allowing or forcing sequential tool execution behavior.
`response_format`	`string` or `object`	Plain text, JSON, or schema-constrained output.
`json_schema`	`object`	Defining the schema for structured output workflows.
`structured_outputs`	`boolean`	Capability signal for reliable schema-constrained output.
`reasoning`	`object`	Provider-specific reasoning configuration.
`reasoning_effort`	`string`	Lowering or increasing reasoning budget.
`reasoning_tokens`	`integer`	Reasoning-specific token limit or accounting field.
`include_reasoning`	`boolean`	Returning reasoning content or summaries where supported.
`service_tier`	`string`	Choosing a supported request tier such as `fast`, `ultrafast`, or `flex`.
`prompt_cache_key`	`string`	Keeping related requests on the provider that holds their prompt cache.
`prompt_cache_options`	`object`	Setting prompt-cache mode and TTL on OpenAI GPT-5.6 and later.
`cache_control`	`object`	Setting a default Claude cache policy, or marking cache breakpoints on content blocks.
`prompt_cache_retention`	`string`	Setting prompt cache retention on OpenAI models before GPT-5.6.
`provider`	`object`	Influencing routing and provider selection.
`provider_options`	`object`	Passing provider-native settings through the gateway.
`meta` / `usage`	`boolean`	Returning extra metadata or usage accounting in the response.
`debug`	`object`	Requesting routing traces and diagnostic payloads.

Endpoint notes

service_tier is supported on the main text request surfaces:

Use ultrafast, fast (or priority where supported), and flex only when the selected model and provider combination supports them. Ultrafast selects the highest-speed supported tier; fast and priority select the same Fast routing and pricing. standard is the default when service_tier is omitted. Batch is not a service_tier value. Batch requests use the separate Batch API. For Anthropic-compatible Messages requests, Anthropic’s native upstream values are auto and standard_only. Phaseo may normalize or map these values across providers while preserving Anthropic-compatible behavior on /v1/messages. If you are using an official Anthropic SDK with a custom base URL pointed at Phaseo, prefer Anthropic-native values on /v1/messages. For normalized cross-provider tier controls such as ultrafast, priority, and flex, or OpenAI’s fast alias, prefer raw HTTP requests or the gateway-native / OpenAI-style text APIs.

Parameter reference

`model`

Selects the gateway model id for the request.

Field	Value
Type	`string`
Required	Yes
Example	`openai/gpt-5-nano`

Use the canonical model id shown on each model page quickstart unless you intentionally want to rely on an accepted alias. Canonical ids are the safest choice for examples, automation, and long-lived integrations.

`models`

Lists optional fallback models, tried in order after model. model is required and is always the primary model. The next entry is tried only when an attempt fails with 429, 500, 502, 503, or 504.

Field	Value
Type	`string[]`
Required	No. A request with `models` must also set `model`.
Limits	1 to 25 entries. Duplicates and entries equal to `model` are ignored.
Example	`["anthropic/claude-sonnet-5-5", "google/gemini-2.5-flash"]`

Each attempt is validated, logged, and billed as its own request. The response model field and routing trace show the model that served the request. Streaming requests fall back only before the first byte, and video and music jobs only at submission. Fallbacks on an attached dynamic route take precedence, presets are accepted as entries, and phaseo/auto cannot be combined with models. Batch, realtime, and council requests reject models with 400. Multipart endpoints accept repeated models fields, models[] fields, or one JSON array. Read Request-level model fallbacks for precedence, validation, streaming, and billing behavior.

`stream`

Returns output incrementally over Server-Sent Events instead of waiting for one final response body.

Field	Value
Type	`boolean`
Default	`false`
Typical values	`true`, `false`

Turn this on for chat UIs, token-by-token rendering, or long responses where early output improves UX. Leave it off when you want one complete JSON response, simpler retries, or easier structured parsing. Notes:

Streaming support varies by endpoint.
Streaming is usually a transport choice, not a quality control.
Tool-calling or structured-output flows may still stream differently by provider.

`temperature`

Controls how random token selection can be.

Field	Value
Type	`number`
Typical range	`0.0` to `2.0` when supported
Default	Provider and model specific
Good starting point	`0.2` to `0.7`

Lower values make output more conservative and repeatable. Higher values increase variety, which can help for brainstorming or creative writing but can also reduce consistency and schema adherence. Good fits:

extraction
classification
JSON or schema output
creative generation

Practical guidance:

Start low for structured tasks.
Change either temperature or top_p first, not both.
High temperature plus aggressive quantization can magnify instability.

`top_p`

Applies nucleus sampling by limiting candidates to the smallest token set whose cumulative probability mass reaches top_p.

Field	Value
Type	`number`
Typical range	`0.0` to `1.0` when supported
Default	Provider and model specific
Good starting point	`0.9` to `1.0`

Lower values make the model choose from a narrower probability mass, which usually produces safer and more focused output. Higher values let the model consider a broader set of tokens. Notes:

Tune top_p when you want a narrower or broader search space without directly changing temperature.
For most applications, moderate temperature and near-1.0 top_p is a reasonable baseline.

`top_k`

Restricts sampling to the top-k candidate tokens at each step on providers that expose it.

Field	Value
Type	`integer`
Typical range	`>= 1` when supported
Default	Provider and model specific

Lower top_k values tighten the model’s choices and can make output more predictable. Higher values widen the candidate pool. Notes:

top_k is not available on every provider.
Treat it as a more explicit token-pool limiter than top_p.

`max_tokens`

Caps output length on endpoints and providers that still use the max_tokens field name.

Field	Value
Type	`integer`
Typical range	`>= 1`
Default	Provider and model specific

Use it to control cost, latency, and truncation risk. If this value is too small, the output can appear incomplete even when the model behaved correctly.

`max_output_tokens`

Caps output length on routes that use max_output_tokens instead of max_tokens.

Field	Value
Type	`integer`
Typical range	`>= 1`
Default	Provider and model specific

This is semantically the same kind of control as max_tokens, but you should send the field name expected by the selected endpoint or SDK surface.

`max_completion_tokens`

Caps output length on newer OpenAI-style text APIs that use max_completion_tokens.

Field	Value
Type	`integer`
Typical range	`>= 1`
Default	Provider and model specific

This is another output-token budget field. Use the endpoint’s expected name rather than mixing output-length aliases in one request shape.

`frequency_penalty`

Discourages repeated tokens in proportion to how often they have already appeared.

Field	Value
Type	`number`
Typical range	Commonly `-2.0` to `2.0` when supported
Default	Usually `0`

Raise this when the model loops, repeats phrases, or overuses the same wording.

`presence_penalty`

Discourages reusing tokens once they have appeared at all, which can help the model explore new topics or wording.

Field	Value
Type	`number`
Typical range	Commonly `-2.0` to `2.0` when supported
Default	Usually `0`

Compared with frequency_penalty, this is usually a broader novelty control rather than a repeat-count control.

`repetition_penalty`

Applies provider-specific anti-repetition behavior outside the classic OpenAI-style penalty fields.

Field	Value
Type	`number`
Typical range	Provider and model specific, often around `0.0` to `2.0`
Default	Provider and model specific

The intent is similar to frequency_penalty and presence_penalty, but the semantics vary more by provider. Treat it as provider-native behavior rather than a universally identical control.

`seed`

Requests deterministic sampling when the upstream provider supports seeded generation.

Field	Value
Type	`integer`
Default	Unset

Use this for debugging, regression testing, and reproducing behavior as closely as the upstream platform allows. Seeded generation improves reproducibility, but exact determinism is not guaranteed across all providers or infrastructure changes.

`stop`

Defines one or more sequences that terminate generation early.

Field	Value
Type	`string` or `string[]`
Default	Unset
Common use	Parser boundaries, template endings, protocol markers

This is useful when you need hard output boundaries, such as stopping before a footer, tool delimiter, or next synthetic section.

`logprobs`

Requests token-level probability metadata where available.

Field	Value
Type	`boolean`
Default	`false`

This is mainly useful for analysis, evaluation, ranking, debugging, and confidence-style workflows. It is not usually needed for standard product responses.

`top_logprobs`

Requests the top alternative candidate tokens for each output position alongside their log probabilities.

Field	Value
Type	`integer`
Typical range	Provider specific, often `0` to `20`
Requires	`logprobs: true`

Use this when you need to inspect alternative token branches rather than only the chosen output token.

`tools`

Declares callable tools or functions for tool-using model workflows.

Field	Value
Type	`array`
Default	Unset

Use the OpenAI-style tool schema unless the endpoint docs say otherwise. Tool declarations describe what the model may call, not whether it must call one.

`tool_choice`

Controls whether the model may call tools automatically, must not call tools, or must use a specific tool.

Field	Value
Type	`string` or `object`
Common values	`none`, `auto`, `required`

Use none when you want content only, auto when the model may decide, and stricter values when downstream orchestration requires a tool call.

`parallel_tool_calls`

Allows or disallows concurrent tool calls on compatible tool-calling APIs.

Field	Value
Type	`boolean`
Default	Endpoint and provider specific

Disable this when downstream systems require strictly sequential execution, ordered side effects, or simpler agent traces.

`response_format`

Requests a particular output format such as plain text, JSON, or schema-constrained responses.

Field	Value
Type	`string` or `object`
Default	Endpoint and provider specific

Exact accepted shapes depend on the endpoint and provider adapter. Use this when you want more than free-form text, especially for JSON responses and structured extraction flows.

`structured_outputs`

Signals support for reliably structured or schema-constrained responses on the selected route and provider set.

Field	Value
Type	`boolean`
Meaning	Capability signal rather than a direct tuning knob

In quickstart tables, this helps you understand whether the selected endpoint and active providers can reliably support structured-output workflows. It is best interpreted as support metadata.

`json_schema`

Supplies the JSON schema used for structured output enforcement on compatible models and endpoints.

Field	Value
Type	`object`
Used with	Structured output or schema-constrained response flows

Use this when your application needs guaranteed fields, typed extraction, or a strict response contract. Keep schemas narrow and task-specific for better adherence.

`reasoning`

Contains provider-specific reasoning configuration for reasoning-capable APIs.

Field	Value
Type	`object`
Default	Unset

Depending on the route, this may include enablement, effort, token budget, verbosity, or whether reasoning content is returned.

`reasoning_effort`

Requests a lower or higher reasoning budget when the endpoint and model expose that control.

Field	Value
Type	`string`
Common values	Provider specific, often values like `minimal`, `low`, `medium`, `high`, `none`
Default	Provider and model specific

Higher effort can improve difficult reasoning tasks at the cost of latency and token usage. Lower effort is often a better fit for faster, cheaper requests.

`reasoning_tokens`

Represents a reasoning-specific token field where supported.

Field	Value
Type	`integer`
Default	Provider and model specific

Depending on the route, this may be a request knob, a limit, or a response accounting field rather than a universally supported request parameter.

`include_reasoning`

Accepted on Chat Completions for compatibility. Reasoning content and summaries are returned whenever the model produces them, whatever this is set to.

Field	Value
Type	`boolean`
Default	`true`

Reasoning payloads can be large and are not available on every model. To reduce reasoning, lower reasoning.effort or set reasoning.enabled to false.

`service_tier`

Selects a supported routing or pricing tier on compatible text APIs.

Field	Value
Type	`string`
Supported values	`standard`, `fast`, `ultrafast`, `priority`, `flex`
Default	`standard`

Use ultrafast, fast (or priority where supported), and flex only when the chosen model and provider combination supports them. Ultrafast selects the highest-speed supported tier; fast and priority select the same Fast routing and pricing. Omit the field to stay on the default standard tier. Phaseo maps these gateway-normalized tier values to provider-native controls internally, so callers can use the same service_tier values across supported text surfaces. Notes:

Batch is a separate API flow, not a service-tier value.
Support varies by endpoint and provider.
OpenAI’s values are also accepted: auto (or null) asks for no particular tier and is treated like omitting the field, and scale is treated as default, the standard tier.

`prompt_cache_key`

Provides a stable cache affinity key for prompt-cache-aware routing.

Field	Value
Type	`string`
Use for	Sticky routing for related cached prompts

Use this when a series of requests share stable prompt prefixes, such as the steps of an agent loop. Phaseo pins the key to the provider and route (deployment or region) that served the first successful request, and keeps later requests with the same key there until that provider fails, Phaseo’s managed quota on it runs out, or your routing policy no longer allows it. Pins are kept per workspace, and their lifetime follows the provider’s cache retention. See Provider affinity. Chat Completions, Responses, and Anthropic Messages all accept it. Phaseo sends a hash of your workspace ID and key upstream, never the raw key, so workspaces that choose the same key do not share a provider cache. Responses echo the key you sent. Providers whose API rejects the field don’t receive it, and sending it never excludes a provider from routing. Phaseo can also derive affinity from request context, but an explicit key is better for long-running conversations, agent sessions, and repeated workflows.

`prompt_cache_options`

Passes prompt-cache controls to OpenAI and Azure OpenAI GPT-5.6 and later, including GPT-6 Astra, to DeepInfra, and to Moonshot AI.

Field	Value
Type	`object`
Common fields	`mode` (`implicit` or `explicit`), `ttl`
OpenAI TTL	`30m`, or `12h` on OpenAI’s Responses API

Use {"mode":"explicit","ttl":"30m"} when you want OpenAI to cache only the breakpoints you place. OpenAI accepts a ttl of 30m (the default) or 12h, and Azure OpenAI only 30m; see 12-hour cache on OpenAI. DeepInfra accepts 5m or 1h and needs prompt_cache_key on every request in the session. Moonshot AI accepts mode: "implicit" and a ttl of 5m or, on Kimi K3, 1h. Phaseo sends the object to those routes. A lifetime the route cannot take, such as 12h on Azure OpenAI or 1h on a Moonshot AI model without a 1-hour price, is removed and reported in the X-Phaseo-Dropped-Params header. Phaseo removes the object for every other provider and for OpenAI models before GPT-5.6, which use prompt_cache_retention instead. provider_options.openai.prompt_cache_options is also accepted; the top-level value wins.

`cache_control`

Sets a default prompt cache policy for Claude, or marks a cache breakpoint on a content block.

Field	Value
Type	`object`
Common fields	`type`, `ttl`, `scope`
Use for	Claude cache policies and explicit cache breakpoints

Top-level cache_control is accepted on Chat Completions, Responses, and Anthropic Messages requests. It applies to Claude on Anthropic, Claude on AWS, Amazon Bedrock, and Google Vertex AI: Phaseo places up to four breakpoints according to scope (all_text, last_user_message, agent, or none). Other providers ignore it. provider_options.anthropic.cache_control is the same setting and wins when both are sent. On a content block, tool definition, or tool result, cache_control marks a cache breakpoint. Phaseo sends it to Claude routes, Alibaba Model Studio, and Venice, converts it to prompt_cache_breakpoint for OpenAI and Azure OpenAI GPT-5.6 and later, and removes it for other providers. OpenAI’s own prompt_cache_breakpoint: {"mode": "explicit"} is accepted on the same blocks and wins when a block has both. Use type: "ephemeral" with a ttl of 5m (the default) or 1h. Anthropic accepts at most four breakpoints per request; when your marks exceed that, Phaseo keeps the first and the latest three. See Prompt Caching for the scopes and what each provider receives.

`prompt_cache_retention`

Sets the prompt cache retention policy on OpenAI and Azure OpenAI models before GPT-5.6.

Field	Value
Type	`string`
Supported values	`in_memory`, `24h`
Use for	OpenAI prompt cache retention

Phaseo sends this field only to OpenAI and Azure OpenAI models before GPT-5.6, and only the models OpenAI lists accept 24h. GPT-5.6 and later use prompt_cache_options.ttl instead, and other providers don’t receive the field. The provider-specific alias provider_options.openai.prompt_cache_retention remains accepted. When both are present, the top-level prompt_cache_retention value takes precedence.

`provider`

Contains routing constraints and provider preferences.

Field	Value
Type	`object`
Use for	Routing rules, provider selection, compliance constraints

Use this when you want to influence which upstream providers may execute the request, how they should be ranked, or what compliance requirements must be satisfied. Common fields include:

Field	Type	Purpose
`order`	`string[]`	Preferred provider order.
`only`	`string[]`	Restrict routing to specific providers.
`ignore`	`string[]`	Exclude specific providers.
`include_alpha`	`boolean`	Allow alpha providers in routing decisions.
`sort`	`string` or `object`	Rank providers, often by `price`, `latency`, or `throughput`.
`required_execution_region`	`string`	Restrict execution to a required region.
`required_data_region`	`string`	Restrict data handling to a required region.
`require_zero_data_retention`	`boolean`	Require providers that meet zero-data-retention constraints.
`max_price`	`object`	Set ceilings for prompt, completion, image, audio, or request costs.
`quantizations`	`string[]`	Require an eligible offer whose catalogue quantization matches one of these values.

quantizations follows the OpenRouter-compatible provider-routing vocabulary and is accepted under either provider or routing. Matching is case-insensitive and ignores spaces, hyphens, and underscores; unambiguous names such as float8/FP8 and bfloat16/BF16 are aliases. Offers without quantization metadata are excluded whenever this filter is present. If no eligible offer matches, the Gateway returns an error with the requested and currently available quantizations instead of silently routing to a different variant.

`provider_options`

Contains provider-specific passthrough settings that should not be normalized into the shared gateway request shape.

Field	Value
Type	`object`
Use for	Provider-native controls

Examples include:

openai.context_management
openai.prompt_cache_retention
anthropic.cache_control
google.cached_content
xai.conversation_id

Use this when you need a provider-native feature but still want the rest of the request to stay on the gateway’s common schema. Prefer top-level cache_control for Claude cache policies and, for OpenAI retention, top-level prompt_cache_options on GPT-5.6 and later or top-level prompt_cache_retention on earlier models. google.cached_content names an explicit Gemini cache. It works only with your own credential: a Google API key on Google AI Studio, or a Vertex AI credential on Vertex AI. On Vertex AI, Phaseo sends the request to the project and location in the cache’s full name (projects/{project}/locations/{location}/cachedContents/{id}); a short name uses the project_id and region in your service-account JSON. With Phaseo’s credentials the request returns 400. google.cache_control and google.cache_ttl are accepted but not sent to Google. For provider prompt caching examples across Chat Completions, Responses, and Anthropic Messages, see Prompt Caching.

`meta`

Requests extra response metadata where supported.

Field	Value
Type	`boolean`
Default	Endpoint specific

Use this when you want additional non-core response metadata for debugging, analytics, or downstream inspection.

`usage`

Requests usage accounting details where supported.

Field	Value
Type	`boolean`
Default	Endpoint specific

This is useful when you want explicit token or usage accounting in the response body rather than only relying on headers or dashboards.

`debug`

Enables controlled request and routing diagnostics.

Field	Value
Type	`object`
Use for	Development and troubleshooting only

Supported debug fields include:

Field	Type	Purpose
`enabled`	`boolean`	Enable debug mode for the request.
`return_upstream_request`	`boolean`	Include the transformed upstream request payload.
`return_upstream_response`	`boolean`	Include upstream response payload where available.
`trace`	`boolean`	Return routing or debug traces.
`trace_level`	`summary` or `full`	Control trace verbosity.

Debug payloads can contain sensitive request context. Use them only in development or tightly controlled environments.

Example request

{
  "model": "openai/gpt-5-nano",
  "input": "Summarize this changelog.",
  "stream": false,
  "temperature": 0.3,
  "max_output_tokens": 300,
  "provider": {
    "order": ["openai", "anthropic"],
    "ignore": ["some-provider"],
    "sort": "latency",
    "required_execution_region": "eu",
    "require_zero_data_retention": true
  },
  "debug": {
    "enabled": true,
    "trace": true,
    "trace_level": "summary"
  }
}

Detailed explanations

If you want the deeper “how should I tune this?” guidance rather than the raw field reference, use these next:

Inference Parameters for practical advice on temperature, top_p, top_k, max token limits, stop sequences, and tuning workflow
Sampling and Decoding for how randomness, penalties, and decoding controls change model behavior

​Unsupported parameters

​Quick lookup

​Endpoint notes

​Parameter reference

​model

​models

​stream

​temperature

​top_p

​top_k

​max_tokens

​max_output_tokens

​max_completion_tokens

​frequency_penalty

​presence_penalty

​repetition_penalty

​seed

​stop

​logprobs

​top_logprobs

​tools

​tool_choice

​parallel_tool_calls

​response_format

​structured_outputs

​json_schema

​reasoning

​reasoning_effort

​reasoning_tokens

​include_reasoning

​service_tier

​prompt_cache_key

​prompt_cache_options

​cache_control

​prompt_cache_retention

​provider

​provider_options

​meta

​usage

​debug

​Example request

​Detailed explanations

​Related pages