AI Model Benchmarks

Short-form factuality benchmark for fact-seeking questions with single verifiable answers. Useful for measuring whether a model answers accurately instead of confidently guessing.

Strong fit for the Reliability Floor factuality dimension, but too narrow to stand alone as a general reliability score.

Benchmark for whether long-form answers stay faithful to supplied documents while still addressing the user request.

Good fit for the Reliability Floor groundedness dimension because it tests source-grounded answers rather than broad world knowledge alone.

Verifiable instruction-following benchmark covering constraints such as word counts, required keywords, and formatting rules.

Useful because it avoids purely subjective judging for a core reliability question: did the model do exactly what it was told?

Long-context benchmark that extends needle-in-haystack into harder retrieval, multi-hop tracing, aggregation, and question-answering tasks.

Use as a retrieval stress test, not as proof of real-world long-context reliability by itself.

Holistic long-context evaluation suite with more application-like tasks including retrieval-augmented generation, citations, summarization, and reranking.

Useful counterweight to simple needle tests because the authors explicitly warn that simple synthetic retrieval does not predict real downstream performance well.

Comprehensive RAG benchmark for dynamic factual question answering with web and knowledge-graph search simulations.

Best treated as a RAG-system reliability source, because scores depend on retrieval setup as well as the base model.

Benchmark for long-term assistant memory across sustained interactions, including extraction, multi-session reasoning, temporal reasoning, knowledge updates, and abstention.

Useful for the Reliability Floor memory dimension, but memory claims need especially careful evidence labels and limitations.

Frequently updated objective benchmark covering math, coding, reasoning, language, instruction following, and data analysis.

Useful for fresh-data discipline and objective grading; only the relevant categories should contribute to the Reliability Floor.

Stanford CRFM evaluation framework and benchmark collection designed to make model comparisons more transparent across scenarios, metrics, prompts, and adapters.

Useful as a provenance model for the Reliability Floor because it treats prompt, harness, scenario, and metric choices as part of the result rather than invisible background detail.

Open benchmark and eval ecosystem focused on robust, reproducible agent testing, with public degradation trackers and historical views for coding agents.

Useful when you want repeated agent and harness measurement rather than a single static benchmark snapshot.

Ranks models by how often they introduce hallucinations when summarising a source document, scored by the open HHEM evaluation model. Complements SimpleQA by measuring grounded-summarisation faithfulness rather than short-form recall.

Promoted from the KOL-4013 link backlog. Useful second hallucination signal for the Reliability Floor factuality dimension.

Measures the energy consumption and efficiency of serving open LLMs, not just their quality. Useful for the operating-envelope view where cost, latency, and energy matter alongside capability.

Promoted from the KOL-4013 link backlog. Fills the efficiency gap the capability-only benchmarks ignore.

A collection of prompts that embed misleading or distracting context to test whether a model reasons from first principles or pattern-matches to a wrong but familiar answer. Probes a failure mode standard reasoning benchmarks miss.

Promoted from the KOL-4013 link backlog. Good adversarial-reasoning signal for the reasoning-floor dimension.

Research organisation tracking the compute, data, cost, and capability trends behind frontier AI. Not a per-model benchmark, but a primary source for the macro trend context the hub cites around model progress.

Promoted from the KOL-4013 link backlog as a cited data source rather than a leaderboard.

Artificial Analysis benchmark measuring both how much factual knowledge a model holds and how often it hallucinates when it does not know. Pairs a knowledge-accuracy score with an explicit hallucination rate, so a confident-but-wrong model scores worse than one that abstains.

Promoted from the KOL-4013 top-priority link batch. Strong complement to SimpleQA and the Vectara leaderboard for the reliability/factuality view.