Every Eval Ever Results Now on Hugging Face Model Pages
Hugging Face
Meta
The EEE (Every Eval Ever) project, launched by the EvalEval Coalition, standardizes AI evaluation reporting using a unified JSON schema. Now integrated with Hugging Face Community Evals, it enables cross-linking between model pages and full evaluation records, with over 229,000 results from 31 formats already collected.
The EEE (Every Eval Ever) project, launched in February 2026 by the EvalEval Coalition, aims to standardize AI evaluation reporting across first and third-party evaluators. It uses a unified JSON schema that records who ran the evaluation, which model, how it was accessed, generation settings, metric definitions, and optional per-sample outputs. Hugging Face launched Community Evals at the same time to decentralize benchmark score reporting on the Hub. The two have now been integrated: contributors can send EEE results to Hugging Face Community Evals via a converter that transforms JSON records into the YAML format Hugging Face expects. The resulting scores appear on model pages with a badge linking back to the full EEE record, including generation config, harness version, and reproducibility notes. The EEE datastore on Hugging Face has grown to approximately 229,000 evaluation results across over 22,000 models and 2,200 benchmarks, drawn from 31 different reporting formats. Reproducing these runs from scratch would cost hundreds of thousands of dollars, highlighting the value of centralized reporting. The converter currently supports four official benchmarks: MMLU-Pro, GPQA, HLE, and GSM8K. Users can submit their EEE records to the datastore, then run the converter to generate YAML previews and open pull requests after review.
- Сокращения
- EEE = Every Eval Ever — Каждая оценка когда-либо
- JSON = JavaScript Object Notation — Нотация объектов JavaScript
- JSONL = JSON Lines — JSON Lines
- YAML = YAML Ain't Markup Language — YAML — не язык разметки
- PR = Pull Request — Запрос на слияние
- CLI = Command Line Interface — Интерфейс командной строки
Source: Hugging Face blog —
original
