HumanEval
Resultados de evaluaciones comparativas de IA y rendimiento de modelos
Tipo: percentage
Resultados registrados
82
Puntuación media
82.66%
Rango de puntuaciones
39.70% - 92.90%
Modelo líder
92.90% - Mistral Small 3.2
Puntuaciones a lo largo del tiempo
Puntuaciones individuales de la evaluación a lo largo del tiempo.
Resultados de modelos
| Organización | Modelos | Fecha del resultado | Mejor puntuación | Información | Autodeclarado | Fuente |
|---|---|---|---|---|---|---|
| Mistral Small 3.2 | 20 jun 2025 | 92,90 % | Pass@5 | Sí | Fuente | |
| o1 mini | 12 sept 2024 | 92,40 % | - | Sí | Fuente | |
| Mistral Large 2.1 | 18 nov 2024 | 92 % | inferred version-family alias from mistral-large-2-2407 | Sí | Fuente | |
| Mistral OCR 2 | 22 may 2025 | 92 % | inferred family alias from mistral-large-2-2407 (score=0.3100; benches=5) | Sí | Fuente | |
| Qwen 2.5 Coder 32B Instruct | - | 91,50 % | inferred high-confidence family alias from qwen2.5-vl-32b (score=0.4641; benches=28) | Sí | Fuente | |
| Qwen 2.5 VL 32B Instruct | 24 mar 2025 | 91,50 % | - | Sí | Fuente | |
| Qwen 2.5 VL 3B Instruct | - | 91,50 % | inferred high-confidence family alias from qwen2.5-vl-32b (score=0.4914; benches=28) | Sí | Fuente | |
| GPT 4o (2024-05-13) | 13 may 2024 | 90,20 % | - | Sí | Fuente | |
| Granite 3.0 8B Instruct | - | 89,73 % | inferred high-confidence family alias from granite-3.3-8b-instruct (score=0.4911; benches=14) | Sí | Fuente | |
| Granite 3.1 8B Instruct | - | 89,73 % | inferred high-confidence family alias from granite-3.3-8b-instruct (score=0.4911; benches=14) | Sí | Fuente | |
| Granite 3.2 8B Instruct | - | 89,73 % | inferred high-confidence family alias from granite-3.3-8b-instruct (score=0.4911; benches=14) | Sí | Fuente | |
| Granite 3.2 8B Instruct Preview | - | 89,73 % | inferred high-confidence family alias from granite-3.3-8b-instruct (score=0.4687; benches=14) | Sí | Fuente | |
| Granite 3.3 2B Instruct | 16 abr 2025 | 89,73 % | inferred family alias from granite-3.3-8b-instruct (score=0.3627; benches=14) | Sí | Fuente | |
| Granite 3.3 8B Instruct | 16 abr 2025 | 89,73 % | - | Sí | Fuente | |
| Granite Guardian 3.0 8B | - | 89,73 % | inferred family alias from granite-3.3-8b-instruct (score=0.4062; benches=14) | Sí | Fuente | |
| Granite Guardian 3.1 8B | - | 89,73 % | inferred family alias from granite-3.3-8b-instruct (score=0.4062; benches=14) | Sí | Fuente | |
| Granite Guardian 3.3 8B | - | 89,73 % | inferred high-confidence family alias from granite-3.3-8b-instruct (score=0.5071; benches=14) | Sí | Fuente | |
| Granite Speech 3.2 8B | - | 89,73 % | inferred family alias from granite-3.3-8b-instruct (score=0.4062; benches=14) | Sí | Fuente | |
| Granite Speech 3.3 8B | - | 89,73 % | inferred high-confidence family alias from granite-3.3-8b-instruct (score=0.5071; benches=14) | Sí | Fuente | |
| Gemini Diffusion | 20 may 2025 | 89,60 % | Pass@1 | Sí | Fuente | |
| DeepSeek V2.5 (2024-09-05) | 05 sept 2024 | 89 % | inferred alias from deepseek-v2.5 | Sí | Fuente | |
| DeepSeek V2.5 (2024-12-10) | 10 dic 2024 | 89 % | inferred alias from deepseek-v2.5 | Sí | Fuente | |
| Granite 4.1 30B | 29 abr 2026 | 88,41 % | pass@1 | Sí | Fuente | |
| LongCat Flash Chat | - | 88,41 % | inferred high-confidence family alias from longcat-flash-chat (score=0.4667; benches=16) | Sí | Fuente | |
| Llama 2 70B Chat | 20 jun 2023 | 88,40 % | inferred family alias from llama-3.3-70b-instruct (score=0.3129; benches=9) | Sí | Fuente |
