> ## Documentation Index
> Fetch the complete documentation index at: https://phaseo.app/docs/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Phaseo Gateway base URL: https://api.phaseo.app/v1. Send `Authorization: Bearer $PHASEO_API_KEY` with every request.
> Model IDs look like `provider/model`, for example `openai/gpt-6-astra`. Do not guess IDs: list them with `GET /v1/models` or check https://phaseo.app/models. `phaseo/free` routes to an available free model and needs no wallet balance.
> For new text integrations prefer `POST /v1/responses`. `POST /v1/chat/completions` (OpenAI format) and `POST /v1/messages` (Anthropic format) suit existing code.
> The full API contract is https://phaseo.app/docs/openapi/v1/openapi.yaml. Official SDKs: `@phaseo/sdk` (TypeScript) and `phaseo` (Python), plus Go, C#, PHP, Ruby and Rust.
> Search these docs over MCP at https://phaseo.app/docs/mcp, and install Phaseo skills with `npx skills add https://phaseo.app/docs`. If a page is wrong or out of date, report it with the MCP `submit_feedback` tool.
> Manage Phaseo accounts with the `phaseo` CLI (`npm install -g @phaseo/cli`, then `phaseo login` or `phaseo login --device-code`), the `/v1` Management API or the MCP server at `https://mcp.phaseo.app/mcp?access=manage`. Do not send users to the dashboard for account tasks. A person is still needed for sign-up, approving a sign-in, paying the first top-up on the checkout link that `phaseo credits add` returns when no card is saved, confirming each saved-card top-up from the emailed link, and account security (password, MFA, passkeys). Create API keys with `phaseo keys create --name <name> --json`; never guess one. Credit top-ups need `phaseo login --scopes billing:write` first. Task map: https://phaseo.app/docs/v1/developers/manage-from-an-agent.md.

# Benchmark Methodology

> Learn how Phaseo collects, normalises, and publishes benchmark results.

Benchmarks provide the foundation for comparisons across models and providers. This page explains how data enters the platform and how we maintain consistency.

***

## Data sources

* **Official releases** — Provider announcements, technical blogs, and press materials.
* **Academic papers** — Peer-reviewed publications and arXiv preprints with reproducible results.
* **Community submissions** — Verified contributions from the Phaseo community and partners.

Every score must include a public citation or reproducible evaluation before it is accepted.

***

## Normalisation

We standardise benchmark names, metrics, and splits to make cross-model comparisons accurate:

1. Map each submission to an internal benchmark ID.
2. Convert reported metrics to a canonical scale (for example percentages).
3. Note any deviations, such as few-shot variants or custom evaluation harnesses.
4. Run automated validation to ensure fields and ranges are sensible.

***

## Update cadence

* **Weekly** for popular suites such as MMLU, GSM8K, and HumanEval.
* **Monthly** for niche or emerging benchmarks.
* **On-demand** when providers or community members submit high-quality evaluations.

Large updates trigger changelog entries so you can audit historical movements.

***

## Transparency and reproducibility

| Practice | Outcome |
| - | - |
| Link to primary sources | Users can verify scores quickly. |
| Track `last_updated` dates | Highlights when a benchmark may need review. |
| Document evaluation setup | Clarifies prompt format, context windows, and sampling. |
| Flag human-reviewed scores | Distinguishes manual validation from automated pipelines. |

***

## Feedback loop

If you notice discrepancies or missing data:

1. Open an issue with the evidence described in [Submit benchmark scores](https://phaseo.app/docs/v1/research/submitting-scores.md).
2. Respond to review questions in the issue. Maintainers publish verified corrections through the database editor.
3. Reach out in Discord to coordinate larger research submissions.

We review every report and provide status updates as the benchmark is verified.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.