> ## Documentation Index
> Fetch the complete documentation index at: https://phaseo.app/docs/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Phaseo Gateway base URL: https://api.phaseo.app/v1. Send `Authorization: Bearer $PHASEO_API_KEY` with every request.
> Model IDs look like `provider/model`, for example `openai/gpt-6-astra`. Do not guess IDs: list them with `GET /v1/models` or check https://phaseo.app/models. `phaseo/free` routes to an available free model and needs no wallet balance.
> For new text integrations prefer `POST /v1/responses`. `POST /v1/chat/completions` (OpenAI format) and `POST /v1/messages` (Anthropic format) suit existing code.
> The full API contract is https://phaseo.app/docs/openapi/v1/openapi.yaml. Official SDKs: `@phaseo/sdk` (TypeScript) and `phaseo` (Python), plus Go, C#, PHP, Ruby and Rust.
> Search these docs over MCP at https://phaseo.app/docs/mcp, and install Phaseo skills with `npx skills add https://phaseo.app/docs`. If a page is wrong or out of date, report it with the MCP `submit_feedback` tool.
> Manage Phaseo accounts with the `phaseo` CLI (`npm install -g @phaseo/cli`, then `phaseo login` or `phaseo login --device-code`), the `/v1` Management API or the MCP server at `https://mcp.phaseo.app/mcp?access=manage`. Do not send users to the dashboard for account tasks. A person is still needed for sign-up, approving a sign-in, paying the first top-up on the checkout link that `phaseo credits add` returns when no card is saved, confirming each saved-card top-up from the emailed link, and account security (password, MFA, passkeys). Create API keys with `phaseo keys create --name <name> --json`; never guess one. Credit top-ups need `phaseo login --scopes billing:write` first. Task map: https://phaseo.app/docs/v1/developers/manage-from-an-agent.md.

# Choosing a Quantization Strategy

> A practical framework for selecting BF16/FP8/INT8/INT4 variants by workload.

Use this workflow to choose quantization variants safely.

To enforce the choice in Phaseo Gateway, send an OpenRouter-compatible routing filter:

```json theme={null}
{
  "model": "meta/llama-3.3-70b-instruct",
  "messages": [{ "role": "user", "content": "Hello" }],
  "provider": {
    "quantizations": ["FP8"]
  }
}
```

The filter applies to provider-model offers, not just provider names. Matching is case-insensitive and supports unambiguous aliases such as `float8` for `FP8` and `bfloat16` for `BF16`. An offer with missing quantization metadata does not qualify. If no eligible offer matches, the request fails with routing diagnostics; Phaseo does not fall back to an unverified quantization. The equivalent `routing.quantizations` shape is also accepted.

## Step 1: define your success criteria

Before choosing a quant:

* Set quality thresholds (task accuracy, pass rates, structured output validity).
* Set latency and cost targets.
* Define unacceptable failure modes.

## Step 2: pick a high-quality baseline

Start with a stronger baseline (often `BF16` or `FP16`) and measure:

* Output quality
* Median and tail latency
* Token/cost profile

This becomes your comparison anchor.

## Step 3: step down gradually

Evaluate progressively lower precision:

1. `FP8` or `INT8`
2. `FP4` or `INT4` only if needed for cost/throughput

Do not skip directly to the smallest variant unless you already have strong evidence it works for your workload.

## Step 4: evaluate real workloads

Use production-like prompts:

* Long prompts
* Multi-step reasoning
* Tool calling
* Edge cases from support tickets

## Step 5: decide per endpoint/use case

One quantization does not need to fit everything.

Examples:

* Customer support and critical decisioning: higher precision.
* Large-volume classification/summarization: lower precision can be acceptable.

## Suggested starting points

| Workload type | Good starting quant |
| - | - |
| Quality-critical reasoning | `BF16` / `FP16` |
| Balanced cost + quality | `FP8` / `INT8` |
| High-throughput, cost-sensitive | `INT4` / `FP4` (after validation) |

## Deployment policy

* Keep a fallback model/quantization configured.
* Monitor quality drift after rollout.
* Re-test when provider runtime or model revision changes.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.