AI Battle > Rankings > MMMU-Pro: Total Cost to Run
Gemini 3.7 Flash (low) by Google leads the MMMU-Pro: Total Cost to Run ranking with a score of 1.68. The ranking covers 19 evaluated eval mmmu from a public dataset that AI Battle re-checks daily.
Last updated: . Source: public benchmark datasets and provider pricing, aggregated daily; 19 entries shown.
| # | Entry | Creator | Value | Δ vs #1 |
|---|---|---|---|---|
| 1 | Gemini 3.7 Flash (low) | 1.68 | 0.0% | |
| 2 | Muse Glimmer (high) | Meta | 0.36 | -78.7% |
| 3 | Gemini 3.5 Flash-Lite | 0.67 | -60.0% | |
| 4 | GPT-5.6 Luna (max) | OpenAI | 0.59 | -64.9% |
| 5 | Gemini 3.7 Flash (high) | 1.68 | 0.0% | |
| 6 | MiniMax-M3 | MiniMax | 0.41 | -75.8% |
| 7 | DeepSeek V4.1 Flash (Reasoning, Max Effort) | DeepSeek | 0.25 | -85.1% |
| 8 | Qwen3.8 27B (xhigh) | Alibaba | 0.49 | -71.2% |
| 9 | GPT-6 Astra (medium) | OpenAI | 10.0 | 493.8% |
| 10 | Gemini 3.8 Flash (high) | 1.68 | 0.0% | |
| 11 | GPT-6 Astra (high) | OpenAI | 10.0 | 493.8% |
| 12 | Inkling (xhigh) | Thinking Machines | 1.76 | 4.7% |
| 13 | Mistral Medium 3.5 | Mistral | 0.50 | -70.6% |
| 14 | GPT-6 Astra (xhigh) | OpenAI | 10.0 | 493.8% |
| 15 | GPT-5.6 Sol (max) | OpenAI | 4.00 | 137.2% |
| 16 | GPT-5.6 Terra (max) | OpenAI | 3.18 | 88.9% |
| 17 | GPT-6 Astra (max) | OpenAI | 10.0 | 493.8% |
| 18 | Claude Opus 5 (Adaptive Reasoning, Max Effort) | Anthropic | 6.04 | 258.3% |
| 19 | Kimi K3 (max) | Kimi | 3.63 | 115.6% |
Across the 19 numeric entries in this window the median is 1.68 and the mean is 3.52, so the typical entry sits 0.00 below the leader. The window spans 0.25 to 10.0.
12 of 19 entries are at or above the median, and the top three hold 45% of the combined value — read it against the metric below, since a higher number is not "better" for every dataset.
10 creators appear in this window; the three most-represented hold 63% of the entries.
| Creator | Entries | Highest value | Top entry |
|---|---|---|---|
| OpenAI | 7 | 10.0 | GPT-6 Astra (medium) |
| 4 | 1.68 | Gemini 3.7 Flash (low) | |
| Anthropic | 1 | 6.04 | Claude Opus 5 (Adaptive Reasoning, Max Effort) |
| Kimi | 1 | 3.63 | Kimi K3 (max) |
| Thinking Machines | 1 | 1.76 | Inkling (xhigh) |
| Mistral | 1 | 0.50 | Mistral Medium 3.5 |
| Alibaba | 1 | 0.49 | Qwen3.8 27B (xhigh) |
| MiniMax | 1 | 0.41 | MiniMax-M3 |
| Meta | 1 | 0.36 | Muse Glimmer (high) |
| DeepSeek | 1 | 0.25 | DeepSeek V4.1 Flash (Reasoning, Max Effort) |
The same models are measured across multiple independent datasets. Aggregating the 19 entries above against every other AI Battle dataset gives this cross-section, computed by AI Battle and published nowhere upstream:
| Model | Datasets | Best placements elsewhere |
|---|---|---|
| Gemini 3.7 Flash (low) | 3 | #5/19 in MMMU-Pro: Token Usage, #7/19 in MMMU-Pro: Score |
| Muse Glimmer (high) | 43 | #1/20 in Output Tokens per Intelligence Index Task, #1/20 in Cost per Intelligence Index Task, #1/20 in Cost per Intelligence Index Task |
| Gemini 3.5 Flash-Lite | 44 | #1/20 in End-to-End Response Time, #1/20 in Output Speed, #1/20 in Time per Intelligence Index Task |
| GPT-5.6 Luna (max) | 58 | #1/11 in Cost per Task, #1/11 in Cost per Task, #3/20 in Pricing: Cache Hit, Input, and Output |
| Gemini 3.7 Flash (high) | 3 | #5/19 in MMMU-Pro: Score, #7/19 in MMMU-Pro: Token Usage |
| MiniMax-M3 | 57 | #1/20 in AA-Omniscience Hallucination Rate, #1/20 in 𝜏³-Banking: Output Tokens per Task, #4/20 in Pricing: Cache Hit, Input, and Output |
| DeepSeek V4.1 Flash (Reasoning, Max Effort) | 56 | #3/20 in End-to-End Response Time, #3/20 in Latency: Time To First Answer Token, #2/11 in Cost per Task |
| Qwen3.8 27B (xhigh) | 43 | #6/20 in AA-Omniscience Hallucination Rate, #6/20 in 𝜏³-Banking: Score, #8/20 in AA-LCR v1.1: Cost per Task |
| GPT-6 Astra (medium) | 12 | #1/20 in AA-Omniscience Index: Token Usage, #1/19 in MMMU-Pro: Token Usage, #2/20 in Terminal-Bench v4.0: Output Tokens per Task |
| Gemini 3.8 Flash (high) | 60 | #1/11 in Speed, #1/11 in Speed, #2/20 in Output Speed |
| GPT-6 Astra (high) | 12 | #1/20 in AA-Omniscience Index: Score, #2/19 in MMMU-Pro: Token Usage, #2/19 in MMMU-Pro: Score |
| Inkling (xhigh) | 52 | #3/20 in Context Window, #3/20 in 𝜏³-Banking: Output Tokens per Task, #5/20 in AA-Briefcase Output Tokens per Task |
| Mistral Medium 3.5 | 41 | #2/20 in GDPval-AA v2: Output Tokens per Task, #3/20 in Output Tokens per Intelligence Index Task, #3/20 in Output Tokens per Intelligence Index Task |
| GPT-6 Astra (xhigh) | 15 | #1/20 in Terminal-Bench v4.0: Score, #2/20 in Humanity's Last Exam: Output Tokens per Task, #3/20 in AA-Omniscience Index: Score |
| GPT-5.6 Sol (max) | 56 | #3/20 in SciCode: Output Tokens per Task, #4/20 in 𝜏³-Banking: Output Tokens per Task, #5/20 in AA-LCR v1.1: Output Tokens per Task |
| GPT-5.6 Terra (max) | 52 | #7/20 in AA-Briefcase Output Tokens per Task, #8/20 in Humanity's Last Exam: Output Tokens per Task, #9/20 in Output Speed |
| GPT-6 Astra (max) | 51 | #1/19 in MMMU-Pro: Score, #2/20 in AA-Omniscience Index, #2/20 in Artificial Analysis Intelligence Index by Open Weights / Proprietary |
| Claude Opus 5 (Adaptive Reasoning, Max Effort) | 44 | #2/20 in AA-Briefcase Elo, #2/20 in GDPval-AA v2 Leaderboard, #2/20 in AA-Briefcase Elo |
| Kimi K3 (max) | 60 | #1/20 in Context Window, #1/20 in AA-LCR v1.1: Score, #1/14 in Model Size: Total and Active Parameters |
This ranking carries the metrics
answerCost, inputCost, reasoningCost from the
MMMU-Pro: Total Cost to Run dataset in the tab::eval-mmmu family. Units and the
direction of "better" are not declared by the source dataset, so AI Battle does not invent
them: values are shown exactly as published, and the ordering is the source dataset's own.
Rebuilt on from the
public dataset MMMU-Pro: Total Cost to Run (family tab::eval-mmmu).
The source lists 19 entries and all 19 are shown here. AI Battle requests
the dataset's rows, keeps the ordering the source publishes, and computes only the statistics
on this page: median, mean, spread, the #1–#2 gap, the creator breakdown and the raw-data file.
Nothing on this page is generated text about the entries — the numbers come from the source.
Refresh cadence: every deploy re-pulls the dataset, so this page tracks the source's own update cycle. Corrections: see the methodology page.
Machine-readable copy of this table, with the statistics below and the source provenance: /data/rankings/eval-mmmu-mmmu-pro-total-cost-to-run.json. Free to reuse with attribution to AI Battle and the source dataset.
Gemini 3.7 Flash (low) by Google currently leads the MMMU-Pro: Total Cost to Run ranking with a score of 1.68, the highest value in this dataset across eval mmmu. AI Battle aggregates the dataset from its public source and re-checks it daily.
The MMMU-Pro: Total Cost to Run ranking is rebuilt daily from the upstream public dataset. Scores are the source dataset's own published values, so positions track whatever that source last released — each ranking page names the dataset it comes from.
The top 3 are: 1. Gemini 3.7 Flash (low) (1.68), 2. Muse Glimmer (high) (0.36), 3. Gemini 3.5 Flash-Lite (0.67).