> ## Documentation Index
> Fetch the complete documentation index at: https://phaseo.app/docs/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Phaseo Gateway base URL: https://api.phaseo.app/v1. Send `Authorization: Bearer $PHASEO_API_KEY` with every request.
> Model IDs look like `provider/model`, for example `openai/gpt-6-astra`. Do not guess IDs: list them with `GET /v1/models` or check https://phaseo.app/models. `phaseo/free` routes to an available free model and needs no wallet balance.
> For new text integrations prefer `POST /v1/responses`. `POST /v1/chat/completions` (OpenAI format) and `POST /v1/messages` (Anthropic format) suit existing code.
> The full API contract is https://phaseo.app/docs/openapi/v1/openapi.yaml. Official SDKs: `@phaseo/sdk` (TypeScript) and `phaseo` (Python), plus Go, C#, PHP, Ruby and Rust.
> Search these docs over MCP at https://phaseo.app/docs/mcp, and install Phaseo skills with `npx skills add https://phaseo.app/docs`. If a page is wrong or out of date, report it with the MCP `submit_feedback` tool.
> Manage Phaseo accounts with the `phaseo` CLI (`npm install -g @phaseo/cli`, then `phaseo login` or `phaseo login --device-code`), the `/v1` Management API or the MCP server at `https://mcp.phaseo.app/mcp?access=manage`. Do not send users to the dashboard for account tasks. A person is still needed for sign-up, approving a sign-in, paying the first top-up on the checkout link that `phaseo credits add` returns when no card is saved, confirming each saved-card top-up from the emailed link, and account security (password, MFA, passkeys). Create API keys with `phaseo keys create --name <name> --json`; never guess one. Credit top-ups need `phaseo login --scopes billing:write` first. Task map: https://phaseo.app/docs/v1/developers/manage-from-an-agent.md.

# Exploring Benchmarks

> Understand how AI model benchmarks are collected, standardised, and visualised on Phaseo.

Benchmarks are at the heart of how Phaseo measures model quality and progress.
They provide **objective, repeatable metrics** that help you compare model performance across tasks, domains, and providers — from reasoning and coding to language understanding and image recognition.

***

## What are benchmarks?

A **benchmark** is a dataset or evaluation designed to test a model’s performance on a specific task.
By running multiple models on the same dataset, we can measure relative strengths and weaknesses.

Common examples include:

* 🧠 **MMLU (Massive Multitask Language Understanding)** — tests general knowledge and reasoning.
* 🔢 **GSM8K** — tests mathematical reasoning and problem-solving.
* 💬 **HellaSwag** — evaluates common sense and contextual understanding.
* 🧮 **ARC-Challenge** — measures scientific reasoning and logic.
* 💡 **HumanEval** — assesses programming and code generation skills.

***

## How Phaseo handles benchmarks

Phaseo collects benchmark data from **multiple public sources** and **official reports**, then standardises them into a consistent format.

Each model’s benchmark section includes:

* **Score value** — typically expressed as a percentage or normalised scale.
* **Dataset source** — e.g. MMLU, GSM8K, etc.
* **Evaluation method** — indicates how the score was derived (official report, third-party eval, or community submission).
* **Last updated** — shows when the benchmark data was last refreshed.

***

## Benchmark categories

Benchmarks are grouped into categories that represent the skill being tested:

| Category | Examples | Description |
| - | - | - |
| **Reasoning** | MMLU, GSM8K, ARC | Tests general logic, multi-step reasoning, and problem-solving. |
| **Language Understanding** | HellaSwag, BoolQ, PIQA | Measures reading comprehension, common sense, and linguistic ability. |
| **Code Generation** | HumanEval, MBPP | Evaluates programming and code synthesis accuracy. |
| **Math & Science** | GSM8K, MATH, SciQ | Focuses on numerical and scientific reasoning. |
| **Multimodal** | MathVista, MMMU | Combines text and visual comprehension tasks. |

***

## How to interpret scores

| Value Range | Interpretation |
| - | - |
| **90-100%** | Exceptional — near state-of-the-art performance. |
| **80-89%** | Excellent — capable for most production use cases. |
| **60-79%** | Good — reliable for general tasks but may struggle on edge cases. |
| **\< 60%** | Developing — best for experimental or research scenarios. |

Keep in mind that some benchmarks have **different scoring systems**.
Phaseo normalises these where possible to make comparisons more meaningful.

***

## Visualising benchmarks on Phaseo

Benchmark results are visualised using:

* 📊 **Bar charts** for comparing models within the same dataset.
* 📈 **Trend charts** showing model improvements over time.
* 🧮 **Aggregate scores** to summarise performance across multiple datasets.

Each benchmark dataset on Phaseo includes filters for:

* Model type (chat, code, embedding, etc.)
* Provider
* Release year
* Dataset category

This lets you quickly identify patterns — like which families dominate reasoning or coding tasks.

***

## Data sources and methodology

Phaseo aggregates benchmark data from:

* Official research papers and model reports.
* Provider evaluation pages (OpenAI Evals, Anthropic Reports, etc.).
* Open community datasets (e.g. Hugging Face leaderboards).
* Direct submissions from users via GitHub.

All data is standardised and verified before being published.
See [Research & Methodology → Benchmark Methodology](https://phaseo.app/docs/v1/research/benchmark-methodology.md) for full details.

***

## Example use cases

| Goal | Example |
| - | - |
| Compare reasoning models | “Which model scores highest on GSM8K?” |
| Evaluate cost-performance trade-offs | “Does GPT-4o justify its higher cost compared to Claude 3.5 Sonnet?” |
| Track yearly progress | “How much has MMLU accuracy improved since 2022?” |
| Research benchmarks by category | “Which models lead in multimodal reasoning?” |

***

## Contributing benchmark data

Open a GitHub issue with evidence for new or corrected benchmark scores.
Each submission is reviewed before inclusion to ensure consistency and accuracy.

> [card] [Contribute Benchmark Data](https://phaseo.app/docs/v1/research/submitting-scores.md)
>
> Learn what evidence to include in a benchmark issue.

***

## Next steps

Once you understand how benchmark data works, you can explore the **API providers** that make these models available programmatically.

> [card] [Explore API Providers](https://phaseo.app/docs/v1/exploring/api-providers.md)
>
> See where and how models are hosted, priced, and accessed.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.