What's new
now jobs Jobs market snapshot refreshed now data Recomputed benchmark-weighted quality scores now data Updated speed measurements now data Synced Chatbot Arena benchmark track now data Validated official pricing snapshots now data Pulled latest OpenRouter price index 25 May digest OpenGuardrails: An Open-Source Context-Aware AI Guardrails Platform 25 May digest Published the 2026-05-25 daily digest 25 May digest Workbench Launches Open Source BullMQ Dashboard For Node Backends 24 May digest SpecBench Tests Reward Hacking In Long Horizon Coding Agents now jobs Jobs market snapshot refreshed now data Recomputed benchmark-weighted quality scores now data Updated speed measurements now data Synced Chatbot Arena benchmark track now data Validated official pricing snapshots now data Pulled latest OpenRouter price index 25 May digest OpenGuardrails: An Open-Source Context-Aware AI Guardrails Platform 25 May digest Published the 2026-05-25 daily digest 25 May digest Workbench Launches Open Source BullMQ Dashboard For Node Backends 24 May digest SpecBench Tests Reward Hacking In Long Horizon Coding Agents

MATH-500

reasoning

MATH-500 evaluates mathematical problem-solving on 500 competition-level problems spanning algebra, geometry, number theory, counting, and probability from AMC, AIME, and other competitions.

View paper / source

36

Models Tested

98.5

Best Score

86.3

Average Score

0–100

Scale Range

1.3x

Weight

How It Works

Models must solve competition-level mathematics problems and produce exact final answers. Problems require multi-step reasoning, creative problem-solving, and precise calculation. Answers are verified exactly — no partial credit.

Why It Matters

Mathematical reasoning is a core capability for AI systems. Competition-level problems require chained logical reasoning that cannot be solved by pattern matching alone, making this a strong test of genuine reasoning ability.

Limitations

Focuses on competition-style maths which may not reflect practical mathematical ability. Models may have seen similar problems in training data. Does not test ability to formulate problems, only solve them.

Leaderboard — MATH-500

# Model Provider Score
🥇 GPT-5.2 Pro OpenAI 98.5
🥈 o3 Pro OpenAI 98.0
🥉 R1 0528 DeepSeek 97.8
4 GPT-5 Pro OpenAI 97.5
5 R1 DeepSeek 97.3
6 o3 OpenAI 96.7
7 o1 OpenAI 96.4
8 o4 Mini OpenAI 96.3
9 Grok 4 xAI 95.0
10 Phi-4 Reasoning Microsoft 94.3
11 o3 Mini OpenAI 94.0
12 Qwen3 Max Alibaba 93.5
13 Qwen3 235B A22B Alibaba 92.0
14 Grok 3 xAI 91.5
15 QwQ 32B Alibaba 90.6
16 Gemini 2.5 Pro Preview 06-05 Google 90.2
17 o1-mini OpenAI 90.0
18 Claude Opus 4 Anthropic 88.7
19 Claude 3.7 Sonnet Anthropic 86.0
20 Claude Sonnet 4 Anthropic 85.4
21 Grok 3 Mini xAI 85.0
22 DeepSeek V3 0324 DeepSeek 84.0
23 GPT-4.1 OpenAI 83.0
24 Gemini 2.5 Flash Google 82.3
25 GPT-4.5 OpenAI 82.0
26 Llama 4 Maverick Meta 81.5
27 Phi 4 Microsoft 80.4
28 Qwen2.5 72B Instruct Alibaba 80.0
29 DeepSeek V3 DeepSeek 78.3
30 Claude 3.5 Sonnet Anthropic 78.3
31 Llama 4 Scout Meta 76.8
32 GPT-4o (2024-05-13) OpenAI 76.6
33 GPT-4o-mini (2024-07-18) OpenAI 70.2
34 Claude 3.5 Haiku Anthropic 69.3
35 Gemini 1.5 Pro Google 67.7
36 Claude 3 Opus Anthropic 60.1
All Benchmarks