Chips · inference benchmark

NVIDIA B200 served 2.8 times as many DeepSeek-R1 tokens per chip

Server scenario: the best NVIDIA B200 result per chip went from 2,324 to 6,462 tokens a second between MLPerf® Inference v5.1 and v6.0. NVIDIA ran entry 5.1-0069 on 8 chips; Nebius ran entry 6.0-0083 on 8.

MLCommons publishes each round of MLPerf® Inference: Datacenter as one result for each whole system. This page divides each result by the system’s count of chips, so that one chip can be set against another and against itself a round later.

inference.csv inference-gains.csv MLPerf® Inference v6.1 published 16 Sept 2026

Embed

The largest gains per chip, in a frame

<iframe src="https://jesseahawkins.com/chips/embed/inference/" title="AI chips on the inference benchmark: NVIDIA B200 served 2.8 times as many DeepSeek-R1 tokens per chip" width="100%" height="560" style="border:0;max-width:100%" loading="lazy"></iframe>

The frame credits MLCommons and links back to its page. Preview it.

Cite

This page

Jesse Hawkins, “AI chips on the inference benchmark: NVIDIA B200 served 2.8 times as many DeepSeek-R1 tokens per chip”, jesseahawkins.com, data retrieved 10 Nov 2026. https://jesseahawkins.com/chips/inference/. Data: MLPerf® Inference: Datacenter results, Closed division, Available category, v5.1, v6.0, v6.1, from the MLCommons results repositories.

BibTeX

@misc{hawkins2026chipsinference,
  author = {Hawkins, Jesse},
  title = {{AI chips on the inference benchmark: NVIDIA B200 served 2.8 times as many DeepSeek-R1 tokens per chip}},
  year = {2026},
  howpublished = {\url{https://jesseahawkins.com/chips/inference/}},
  note = {Data: MLPerf® Inference: Datacenter results, Closed division, Available category, v5.1, v6.0, v6.1, from the MLCommons results repositories, retrieved 2026-11-10}
}

The record · this site

jesseahawkins.com, “AI chips on the inference benchmark”, from MLPerf® Inference: Datacenter results v5.1, v6.0, v6.1, retrieved 10 Nov 2026. https://jesseahawkins.com/chips/inference/

The trademark

The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

Download

Post on X

The same chip, one round later

the best result per chip in each round, where MLCommons marks the two versions compatible

NVIDIA B200 DeepSeek-R1 · Server · v5.1 to v6.0 2.8× times the result per chip of the round before
NVIDIA GB300 DeepSeek-R1 · Server · v5.1 to v6.0 2.8× times the result per chip of the round before
NVIDIA GB200 DeepSeek-R1 · Server · v5.1 to v6.0 2.1× times the result per chip of the round before
NVIDIA GB200 DeepSeek-R1 · Offline · v5.1 to v6.0 1.8× times the result per chip of the round before
NVIDIA B200 DeepSeek-R1 · Offline · v5.1 to v6.0 1.8× times the result per chip of the round before
NVIDIA GB300 DeepSeek-R1 · Offline · v5.1 to v6.0 1.7× times the result per chip of the round before
NVIDIA L40S Whisper · Offline · v5.1 to v6.0 1.6× times the result per chip of the round before
NVIDIA GB300 Llama 3.1 405B · Server · v5.1 to v6.0 1.5× times the result per chip of the round before
NVIDIA GB200 Llama 3.1 405B · Interactive · v5.1 to v6.0 1.4× times the result per chip of the round before
NVIDIA GB200 Llama 2 70B, 99% accuracy · Interactive · v5.1 to v6.0 1.4× times the result per chip of the round before
NVIDIA GB200 Llama 2 70B, 99.9% accuracy · Interactive · v5.1 to v6.0 1.4× times the result per chip of the round before
NVIDIA RTX PRO 6000 Blackwell Llama 3.1 8B · Interactive · v5.1 to v6.0 1.2× times the result per chip of the round before

Every gain, with the two entries behind it

Each chip’s best result per chip in two rounds of MLPerf® Inference: Datacenter, Closed division, Available category
Chip	Benchmark	Scenario	Before, per chip	After, per chip	Change
NVIDIA B200	DeepSeek-R1	Server	2,324v5.1 · NVIDIA · 8 chips · 5.1-0069	6,462v6.0 · Nebius · 8 chips · 6.0-0083	2.8×
NVIDIA GB300	DeepSeek-R1	Server	2,907v5.1 · NVIDIA · 72 chips · 5.1-0072	8,064v6.0 · Nebius · 8 chips · 6.0-0082	2.8×
NVIDIA GB200	DeepSeek-R1	Server	2,327v5.1 · NVIDIA · 72 chips · 5.1-0071	4,868v6.0 · CoreWeave · 64 chips · 6.0-0015	2.1×
NVIDIA GB200	DeepSeek-R1	Offline	4,024v5.1 · NVIDIA · 72 chips · 5.1-0071	7,323v6.0 · CoreWeave · 64 chips · 6.0-0015	1.8×
NVIDIA B200	DeepSeek-R1	Offline	4,172v5.1 · NVIDIA · 8 chips · 5.1-0069	7,323v6.0 · Nebius · 8 chips · 6.0-0083	1.8×
NVIDIA GB300	DeepSeek-R1	Offline	5,842v5.1 · NVIDIA · 72 chips · 5.1-0072	9,821v6.0 · CoreWeave · 8 chips · 6.0-0017	1.7×
NVIDIA L40S	Whisper	Offline	1,159v5.1 · Dell · 8 chips · 5.1-0022	1,823v6.0 · RedHat · 2 chips · 6.0-0090	1.6×
NVIDIA GB300	Llama 3.1 405B	Server	170v5.1 · NVIDIA · 72 chips · 5.1-0072	259v6.0 · NVIDIA · 72 chips · 6.0-0078	1.5×
NVIDIA GB200	Llama 3.1 405B	Interactive	138v5.1 · NVIDIA · 72 chips · 5.1-0071	195v6.0 · NVIDIA · 72 chips · 6.0-0075	1.4×
NVIDIA GB200	Llama 2 70B, 99% accuracy	Interactive	7,436v5.1 · NVIDIA · 4 chips · 5.1-0070	10,484v6.0 · NVIDIA · 72 chips · 6.0-0075	1.4×
NVIDIA GB200	Llama 2 70B, 99.9% accuracy	Interactive	7,436v5.1 · NVIDIA · 4 chips · 5.1-0070	10,484v6.0 · NVIDIA · 72 chips · 6.0-0075	1.4×
NVIDIA RTX PRO 6000 Blackwell	Llama 3.1 8B	Interactive	4,507v5.1 · HPE · 8 chips · 5.1-0051	5,511v6.0 · HPE · 8 chips · 6.0-0047	1.2×
NVIDIA GB200	Llama 3.1 405B	Server	161v5.1 · NVIDIA · 72 chips · 5.1-0071	196v6.0 · NVIDIA · 72 chips · 6.0-0075	1.2×
NVIDIA GB300	Llama 3.1 405B	Offline	224v5.1 · NVIDIA · 72 chips · 5.1-0072	271v6.0 · NVIDIA · 72 chips · 6.0-0078	1.2×
Intel Arc Pro B60 Dual 48G	Llama 3.1 8B	Server	1,337v5.1 · Intel · 4 chips · 5.1-0056	1,573v6.0 · Intel · 4 chips · 6.0-0060	1.2×
NVIDIA RTX PRO 6000 Blackwell	Llama 2 70B, 99% accuracy	Offline	3,276v5.1 · HPE · 8 chips · 5.1-0051	3,739v6.0 · HPE · 8 chips · 6.0-0047	1.1×
NVIDIA RTX PRO 6000 Blackwell	Llama 2 70B, 99.9% accuracy	Offline	3,276v5.1 · HPE · 8 chips · 5.1-0051	3,738v6.0 · HPE · 8 chips · 6.0-0047	1.1×
NVIDIA RTX PRO 6000 Blackwell	Llama 2 70B, 99% accuracy	Server	3,251v5.1 · HPE · 8 chips · 5.1-0051	3,591v6.0 · HPE · 8 chips · 6.0-0047	1.1×
NVIDIA RTX PRO 6000 Blackwell	Llama 2 70B, 99.9% accuracy	Server	3,250v5.1 · HPE · 8 chips · 5.1-0051	3,590v6.0 · HPE · 8 chips · 6.0-0047	1.1×
NVIDIA B200	Llama 3.1 8B	Offline	18,370v5.1 · Lenovo · 8 chips · 5.1-0062	20,050v6.0 · Lambda_SIT · 8 chips · 6.0-0062	1.1×
NVIDIA B200	Whisper	Offline	5,667v5.1 · NVIDIA · 8 chips · 5.1-0069	6,166v6.0 · Google · 8 chips · 6.0-0040	1.1×
Intel Arc Pro B60 Dual 48G	Llama 2 70B, 99% accuracy	Offline	752v5.1 · Intel · 4 chips · 5.1-0056	818v6.0 · Intel · 4 chips · 6.0-0060	1.1×
Intel Arc Pro B60 Dual 48G	Llama 2 70B, 99.9% accuracy	Offline	752v5.1 · Intel · 4 chips · 5.1-0056	818v6.0 · Intel · 4 chips · 6.0-0060	1.1×
NVIDIA RTX PRO 6000 Blackwell	Llama 3.1 8B	Offline	5,855v5.1 · HPE · 8 chips · 5.1-0051	6,248v6.0 · HPE · 2 chips · 6.0-0052	1.1×
NVIDIA RTX PRO 6000 Blackwell	Mixtral 8x7B	Server	3,767v5.1 · HPE · 8 chips · 5.1-0051	4,019v6.0 · HPE · 8 chips · 6.0-0046	1.1×
NVIDIA RTX PRO 6000 Blackwell	Llama 3.1 8B	Server	5,758v5.1 · HPE · 8 chips · 5.1-0051	6,099v6.0 · HPE · 8 chips · 6.0-0047	1.1×
NVIDIA H200	Whisper	Offline	4,306v5.1 · HPE · 8 chips · 5.1-0049	4,549v6.0 · RedHat · 8 chips · 6.0-0093	1.1×
NVIDIA B200	Llama 3.1 8B	Interactive	15,284v5.1 · Supermicro · 8 chips · 5.1-0086	16,094v6.0 · Lambda_SIT · 8 chips · 6.0-0062	1.1×
NVIDIA H200 NVL	Llama 3.1 8B	Offline	6,347v5.1 · Cisco · 8 chips · 5.1-0011	6,649v6.0 · Dell · 8 chips · 6.0-0021	1.0×
Intel Arc Pro B60 Dual 48G	Llama 3.1 8B	Offline	1,618v5.1 · Intel · 4 chips · 5.1-0056	1,691v6.0 · Intel · 4 chips · 6.0-0060	1.0×
Intel Arc Pro B60 Dual 48G	Llama 2 70B, 99% accuracy	Server	528v5.1 · Intel · 4 chips · 5.1-0056	550v6.0 · Intel · 4 chips · 6.0-0060	1.0×
Intel Arc Pro B60 Dual 48G	Llama 2 70B, 99.9% accuracy	Server	528v5.1 · Intel · 4 chips · 5.1-0056	550v6.0 · Intel · 4 chips · 6.0-0060	1.0×
NVIDIA RTX PRO 6000 Blackwell	Mixtral 8x7B	Offline	4,092v5.1 · HPE · 8 chips · 5.1-0051	4,205v6.0 · HPE · 8 chips · 6.0-0046	1.0×
AMD Instinct MI300X	Llama 2 70B, 99% accuracy	Server	3,108v5.1 · MangoBoost · 8 chips · 5.1-0066	3,183v6.0 · MangoBoost · 8 chips · 6.0-0069	1.0×
AMD Instinct MI300X	Llama 2 70B, 99.9% accuracy	Server	3,108v5.1 · MangoBoost · 8 chips · 5.1-0066	3,183v6.0 · MangoBoost · 8 chips · 6.0-0069	1.0×
NVIDIA H200 NVL	Llama 2 70B, 99% accuracy	Offline	3,908v5.1 · Dell · 8 chips · 5.1-0023	4,001v6.0 · Dell · 8 chips · 6.0-0021	1.0×
NVIDIA H200 NVL	Llama 2 70B, 99.9% accuracy	Offline	3,908v5.1 · Dell · 8 chips · 5.1-0023	4,001v6.0 · Dell · 8 chips · 6.0-0021	1.0×
NVIDIA B200	Llama 2 70B, 99% accuracy	Server	12,701v5.1 · Nebius · 8 chips · 5.1-0073	12,954v6.0 · HPE · 8 chips · 6.0-0048	1.0×
NVIDIA B200	Llama 2 70B, 99.9% accuracy	Server	12,701v5.1 · Nebius · 8 chips · 5.1-0073	12,953v6.0 · HPE · 8 chips · 6.0-0048	1.0×
NVIDIA B200	Llama 3.1 8B	Server	16,099v5.1 · Dell · 8 chips · 5.1-0028	16,409v6.0 · HPE · 8 chips · 6.0-0048	1.0×
NVIDIA B200	Llama 2 70B, 99.9% accuracy	Offline	12,864v5.1 · Lenovo · 8 chips · 5.1-0061	13,072v6.0 · HPE · 8 chips · 6.0-0048	1.0×
NVIDIA B200	Llama 2 70B, 99% accuracy	Offline	12,864v5.1 · Lenovo · 8 chips · 5.1-0061	12,994v6.0 · HPE · 8 chips · 6.0-0048	1.0×
NVIDIA H200 NVL	Whisper	Offline	3,963v5.1 · HPE · 2 chips · 5.1-0052	3,996v6.0 · Dell · 8 chips · 6.0-0021	1.0×
NVIDIA GB200	Llama 3.1 405B	Offline	214v5.1 · Nebius · 4 chips · 5.1-0074	215v6.0 · NVIDIA · 72 chips · 6.0-0075	1.0×
NVIDIA H200 NVL	Llama 2 70B, 99% accuracy	Server	3,634v5.1 · Dell · 8 chips · 5.1-0023	3,636v6.0 · Dell · 8 chips · 6.0-0021	1.0×
NVIDIA H200 NVL	Llama 2 70B, 99.9% accuracy	Server	3,634v5.1 · Dell · 8 chips · 5.1-0023	3,636v6.0 · Dell · 8 chips · 6.0-0021	1.0×
NVIDIA H200 NVL	Llama 3.1 8B	Server	6,438v5.1 · Quanta_Cloud_Technology · 4 chips · 5.1-0080	6,433v6.0 · Cisco · 4 chips · 6.0-0013	1.0×
NVIDIA H200 NVL	Llama 2 70B, 99% accuracy	Interactive	2,052v5.1 · Dell · 8 chips · 5.1-0023	2,043v6.0 · Dell · 8 chips · 6.0-0021	1.0×
NVIDIA H200 NVL	Llama 2 70B, 99.9% accuracy	Interactive	2,052v5.1 · Dell · 8 chips · 5.1-0023	2,043v6.0 · Dell · 8 chips · 6.0-0021	1.0×
AMD Instinct MI325X	Llama 2 70B, 99% accuracy	Server	4,017v5.1 · GigaComputing · 8 chips · 5.1-0045	3,986v6.0 · MangoBoost · 8 chips · 6.0-0070	1.0×
AMD Instinct MI325X	Llama 2 70B, 99.9% accuracy	Server	4,017v5.1 · GigaComputing · 8 chips · 5.1-0045	3,986v6.0 · MangoBoost · 8 chips · 6.0-0070	1.0×
AMD Instinct MI300X	Llama 2 70B, 99% accuracy	Offline	3,482v5.1 · MangoBoost · 8 chips · 5.1-0066	3,450v6.0 · MangoBoost · 8 chips · 6.0-0069	1.0×
AMD Instinct MI300X	Llama 2 70B, 99.9% accuracy	Offline	3,482v5.1 · MangoBoost · 8 chips · 5.1-0066	3,450v6.0 · MangoBoost · 8 chips · 6.0-0069	1.0×
NVIDIA GB200	Whisper	Offline	5,568v5.1 · Oracle · 4 chips · 5.1-0077	5,513v6.0 · Google · 4 chips · 6.0-0041	1.0×
NVIDIA B200	Llama 2 70B, 99% accuracy	Interactive	7,856v5.1 · GigaComputing · 8 chips · 5.1-0046	7,706v6.0 · HPE · 8 chips · 6.0-0048	1.0×
AMD Instinct MI325X	Llama 2 70B, 99% accuracy	Offline	4,319v5.1 · AMD · 8 chips · 5.1-0003	4,235v6.0 · MangoBoost · 8 chips · 6.0-0070	1.0×
AMD Instinct MI325X	Llama 2 70B, 99.9% accuracy	Offline	4,319v5.1 · AMD · 8 chips · 5.1-0003	4,235v6.0 · MangoBoost · 8 chips · 6.0-0070	1.0×
NVIDIA B200	Llama 2 70B, 99.9% accuracy	Interactive	7,856v5.1 · GigaComputing · 8 chips · 5.1-0046	7,663v6.0 · HPE · 8 chips · 6.0-0048	1.0×
NVIDIA GB200	Llama 2 70B, 99.9% accuracy	Offline	12,934v5.1 · NVIDIA · 4 chips · 5.1-0070	12,334v6.0 · NVIDIA · 72 chips · 6.0-0075	1.0×
NVIDIA GB200	Llama 2 70B, 99% accuracy	Offline	13,015v5.1 · Azure · 4 chips · 5.1-0008	12,334v6.0 · NVIDIA · 72 chips · 6.0-0075	0.9×
NVIDIA GB200	Llama 2 70B, 99% accuracy	Server	12,340v5.1 · NVIDIA · 4 chips · 5.1-0070	11,251v6.0 · NVIDIA · 72 chips · 6.0-0075	0.9×
NVIDIA GB200	Llama 2 70B, 99.9% accuracy	Server	12,340v5.1 · NVIDIA · 4 chips · 5.1-0070	11,251v6.0 · NVIDIA · 72 chips · 6.0-0075	0.9×
NVIDIA B200	Llama 3.1 405B	Offline	208v5.1 · Dell · 8 chips · 5.1-0028	169v6.0 · Google · 8 chips · 6.0-0040	0.8×
NVIDIA B200	Llama 3.1 405B	Server	160v5.1 · Nebius · 8 chips · 5.1-0073	123v6.0 · Google · 8 chips · 6.0-0040	0.8×

Each figure is a result per chip, in its benchmark’s unit: tokens a second for a language model, samples or queries a second for the rest. Two results of one chip can come from different submitters, systems and chip counts, and each row names them. Rounds are compared only where the MLCommons compatibility table marks the benchmark in both versions. That table lists versions to v6.0, so no later round is compared yet.

Each chip’s best in MLPerf® Inference v6.1

published 16 Sept 2026 · 95 systems

The leaders per chip

DeepSeek-R1

Offline · tokens a second per chip

NVIDIA GB300 Oracle · NVIDIA GB300 NVL72 (8x GB300-288GB_aarch64, TensorRT) · 8 chips · entry 6.1-0088 9,629 tokens a second per chip
NVIDIA B300 GigaComputing · G894-AD3-AAX7 · 8 chips · entry 6.1-0045 9,056 tokens a second per chip
NVIDIA GB200 Azure · Azure GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT) · 72 chips · entry 6.1-0009 7,529 tokens a second per chip
NVIDIA B200 CoreWeave · CoreWeave B200 SXM (8x B200-SXM-180GB) · 8 chips · entry 6.1-0021 7,364 tokens a second per chip
AMD Instinct MI355X Crusoe · Crusoe Cloud 64x8 MI355X (512 GPU) — DeepSeek-R1 / SGLang · 512 chips · entry 6.1-0026 5,668 tokens a second per chip

Server · tokens a second per chip

NVIDIA B300 GigaComputing · G894-ZD3-AAX7 · 8 chips · entry 6.1-0046 8,846 tokens a second per chip
NVIDIA GB300 Oracle · NVIDIA GB300 NVL72 (8x GB300-288GB_aarch64, TensorRT) · 8 chips · entry 6.1-0088 8,517 tokens a second per chip
NVIDIA B200 CoreWeave · CoreWeave B200 SXM (8x B200-SXM-180GB) · 8 chips · entry 6.1-0021 7,015 tokens a second per chip
NVIDIA GB200 Azure · Azure GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT) · 72 chips · entry 6.1-0009 5,928 tokens a second per chip
AMD Instinct MI355X Crusoe · Crusoe Cloud 64x8 MI355X (512 GPU) — DeepSeek-R1 / SGLang · 512 chips · entry 6.1-0026 4,698 tokens a second per chip

Interactive · tokens a second per chip

NVIDIA GB300 Azure · Azure GB300 NVL72 (72x GB300-288GB_aarch64 TensorRT) · 72 chips · entry 6.1-0010 3,612 tokens a second per chip
NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT) · 72 chips · entry 6.1-0072 3,337 tokens a second per chip

GPT-OSS-120B

Offline · tokens a second per chip

NVIDIA GB300 CoreWeave · CoreWeave GB300 NVL72 (72x GB300-288GB_aarch64) · 72 chips · entry 6.1-0025 16,635 tokens a second per chip
NVIDIA GB200 RedHat_Supermicro · SMC Lab OpenShift GB200-NVL4 4xGB200-186GB · 4 chips · entry 6.1-0094 15,301 tokens a second per chip
AMD Instinct MI355X AMD · smci355-ccs-aus-m13-13 (AS -4126GS-NMR-LCC) · 8 chips · entry 6.1-0003 15,227 tokens a second per chip
NVIDIA B300 HPE · HPE Compute XD690 (8x NVIDIA B300-SXM-270GB, TensorRT) · 8 chips · entry 6.1-0052 14,321 tokens a second per chip
AMD Instinct MI350X MiTAC · G8825Z5 · 8 chips · entry 6.1-0068 12,127 tokens a second per chip
NVIDIA B200 CoreWeave · CoreWeave B200 SXM (8x B200-SXM-180GB) · 8 chips · entry 6.1-0021 11,436 tokens a second per chip
AMD Instinct MI350P Dell_AMD · XE7745_8xMI350P_2xEPYC_9455 · 8 chips · entry 6.1-0040 6,121 tokens a second per chip
AMD Instinct MI300X MangoBoost · 8xMI300X_2xEPYC_9534 · 8 chips · entry 6.1-0067 3,775 tokens a second per chip

Server · tokens a second per chip

NVIDIA GB300 CoreWeave · CoreWeave GB300 NVL72 (72x GB300-288GB_aarch64) · 72 chips · entry 6.1-0025 16,118 tokens a second per chip
NVIDIA B300 HPE · HPE Compute XD690 (8x NVIDIA B300-SXM-270GB, TensorRT) · 8 chips · entry 6.1-0052 14,211 tokens a second per chip
AMD Instinct MI355X AMD · smci355-ccs-aus-m13-13 (AS -4126GS-NMR-LCC) · 8 chips · entry 6.1-0003 14,155 tokens a second per chip
NVIDIA GB200 RedHat_Supermicro · SMC Lab OpenShift GB200-NVL4 4xGB200-186GB · 4 chips · entry 6.1-0094 12,774 tokens a second per chip
NVIDIA B200 CoreWeave · CoreWeave B200 SXM (8x B200-SXM-180GB) · 8 chips · entry 6.1-0021 11,276 tokens a second per chip
AMD Instinct MI350X MiTAC · G8825Z5 · 8 chips · entry 6.1-0068 10,944 tokens a second per chip
AMD Instinct MI350P Dell_AMD · XE7745_8xMI350P_2xEPYC_9455 · 8 chips · entry 6.1-0040 5,108 tokens a second per chip
AMD Instinct MI300X MangoBoost · 8xMI300X_2xEPYC_9534 · 8 chips · entry 6.1-0067 3,108 tokens a second per chip

Interactive · tokens a second per chip

NVIDIA GB300 NVIDIA · NVIDIA GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT) · 72 chips · entry 6.1-0074 9,296 tokens a second per chip
NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT) · 72 chips · entry 6.1-0072 8,683 tokens a second per chip
AMD Instinct MI355X MangoBoost · MangoBoost 2x8 MI355X (16x MI355X, LLMBoost P/D Disaggregated Interactive Engine) · 16 chips · entry 6.1-0066 3,706 tokens a second per chip

Llama 2 70B, 99.9% accuracy

Offline · tokens a second per chip

NVIDIA GB300 Oracle · NVIDIA GB300 NVL72 (8x GB300-288GB_aarch64, TensorRT) · 8 chips · entry 6.1-0088 16,532 tokens a second per chip
NVIDIA B300 VibeHPC · B300-SXM-270GBx1_TRT · 1 chip · entry 6.1-0103 14,637 tokens a second per chip
AMD Instinct MI355X AMD · smci355-ccs-aus-m13-13 (AS -4126GS-NMR-LCC) · 8 chips · entry 6.1-0003 13,315 tokens a second per chip
NVIDIA B200 CoreWeave · CoreWeave B200 SXM (8x B200-SXM-180GB) · 8 chips · entry 6.1-0021 12,838 tokens a second per chip
NVIDIA GB200 CoreWeave · CoreWeave GB200 NVL72 (72x GB200-186GB_aarch64) · 72 chips · entry 6.1-0023 12,417 tokens a second per chip
AMD Instinct MI350X MiTAC · G8825Z5 · 8 chips · entry 6.1-0068 10,502 tokens a second per chip
AMD Instinct MI350P Dell_AMD · XE7745_8xMI350P_2xEPYC_9455 · 8 chips · entry 6.1-0040 5,286 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell MiTAC · MiTAC_G4520G6_RTX_PRO_6000_PCIE_96GBx8 · 8 chips · entry 6.1-0070 3,754 tokens a second per chip

Server · tokens a second per chip

NVIDIA GB300 Oracle · NVIDIA GB300 NVL72 (8x GB300-288GB_aarch64, TensorRT) · 8 chips · entry 6.1-0088 15,998 tokens a second per chip
NVIDIA B300 VibeHPC · B300-SXM-270GBx1_TRT · 1 chip · entry 6.1-0103 14,496 tokens a second per chip
AMD Instinct MI355X MiTAC · G4826 · 8 chips · entry 6.1-0069 13,068 tokens a second per chip
NVIDIA B200 CoreWeave · CoreWeave B200 SXM (8x B200-SXM-180GB) · 8 chips · entry 6.1-0021 12,800 tokens a second per chip
NVIDIA GB200 CoreWeave · CoreWeave GB200 NVL72 (72x GB200-186GB_aarch64) · 72 chips · entry 6.1-0023 12,648 tokens a second per chip
AMD Instinct MI350X MiTAC · G8825Z5 · 8 chips · entry 6.1-0068 10,626 tokens a second per chip
AMD Instinct MI350P Dell_AMD · XE7745_8xMI350P_2xEPYC_9455 · 8 chips · entry 6.1-0040 5,229 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell Cisco · Cisco UCS C845A M8 (8x RTXPro6000-PCIe-96GB, TensorRT) · 8 chips · entry 6.1-0013 3,665 tokens a second per chip

Interactive · tokens a second per chip

AMD Instinct MI355X MiTAC · G4826 · 8 chips · entry 6.1-0069 9,298 tokens a second per chip
AMD Instinct MI350X MiTAC · G8825Z5 · 8 chips · entry 6.1-0068 7,084 tokens a second per chip
AMD Instinct MI350P Dell_AMD · XE7745_8xMI350P_2xEPYC_9455 · 8 chips · entry 6.1-0040 2,858 tokens a second per chip

Llama 3.1 8B

Offline · tokens a second per chip

NVIDIA B300 Cisco · Cisco UCS C880A M8 (8x NVIDIA B300-SXM-270GB, TensorRT) · 8 chips · entry 6.1-0014 21,389 tokens a second per chip
AMD Instinct MI355X MiTAC · G4826 · 8 chips · entry 6.1-0069 19,910 tokens a second per chip
AMD Instinct MI350X MiTAC · G8825Z5 · 8 chips · entry 6.1-0068 16,243 tokens a second per chip
AMD Instinct MI350P Dell_AMD · XE7745_8xMI350P_2xEPYC_9455 · 8 chips · entry 6.1-0040 9,205 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant Compute DL380a Gen12 (8x RTX Pro 6000 Blackwell Server Edition 96GB, TensorRT) · 8 chips · entry 6.1-0049 6,371 tokens a second per chip
NVIDIA RTX PRO 4500 Blackwell Cisco · Cisco UCS C240 M8 - (5x NVIDIA RTX PRO 4500 Blackwell-32GB, TensorRT) · 5 chips · entry 6.1-0011 2,404 tokens a second per chip
Intel Arc Pro B70 Quanta_Cloud_Technology · 1-node-4x-BMG-B70 · 4 chips · entry 6.1-0090 1,244 tokens a second per chip

Server · tokens a second per chip

NVIDIA B300 Cisco · Cisco UCS C880A M8 (8x NVIDIA B300-SXM-270GB, TensorRT) · 8 chips · entry 6.1-0014 21,744 tokens a second per chip
AMD Instinct MI355X MiTAC · G4826 · 8 chips · entry 6.1-0069 18,890 tokens a second per chip
AMD Instinct MI350X MiTAC · G8825Z5 · 8 chips · entry 6.1-0068 15,213 tokens a second per chip
AMD Instinct MI350P Dell_AMD · XE7745_8xMI350P_2xEPYC_9455 · 8 chips · entry 6.1-0040 9,038 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant Compute DL380a Gen12 (8x RTX Pro 6000 Blackwell Server Edition 96GB, TensorRT) · 8 chips · entry 6.1-0049 6,190 tokens a second per chip
NVIDIA RTX PRO 4500 Blackwell Cisco · Cisco UCS X210 M8 - X580P PCIe Node (4x NVIDIA RTX PRO 4500 Blackwell-32GB, TensorRT) · 4 chips · entry 6.1-0017 2,319 tokens a second per chip
Intel Arc Pro B70 Supermicro · 1-node-4x-BMG-B70 · 4 chips · entry 6.1-0098 1,052 tokens a second per chip

Interactive · tokens a second per chip

AMD Instinct MI355X MiTAC · G4826 · 8 chips · entry 6.1-0069 17,125 tokens a second per chip
NVIDIA B300 Cisco · Cisco UCS C880A M8 (8x NVIDIA B300-SXM-270GB, TensorRT) · 8 chips · entry 6.1-0014 16,146 tokens a second per chip
AMD Instinct MI350X MiTAC · G8825Z5 · 8 chips · entry 6.1-0068 13,046 tokens a second per chip
AMD Instinct MI350P Dell_AMD · XE7745_8xMI350P_2xEPYC_9455 · 8 chips · entry 6.1-0040 7,840 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell Cisco · Cisco UCS C845A M8 (8x RTXPro6000-PCIe-96GB, TensorRT) · 8 chips · entry 6.1-0013 5,771 tokens a second per chip
NVIDIA RTX PRO 4500 Blackwell Cisco · Cisco UCS C845A M8 (8x NVIDIA RTX PRO 4500 Blackwell-32GB, TensorRT) · 8 chips · entry 6.1-0012 1,790 tokens a second per chip

Whisper

Offline · samples a second per chip

NVIDIA B300 VibeHPC · B300-SXM-270GBx1_TRT · 1 chip · entry 6.1-0103 7,059 samples a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant Compute DL380a Gen12 (8x RTX Pro 6000 Blackwell Server Edition 96GB, TensorRT) · 8 chips · entry 6.1-0049 2,410 samples a second per chip
Intel Arc Pro B70 Supermicro · 1-node-4x-BMG-B70 · 4 chips · entry 6.1-0098 1,464 samples a second per chip
NVIDIA RTX PRO 4500 Blackwell Cisco · Cisco UCS C845A M8 (8x NVIDIA RTX PRO 4500 Blackwell-32GB, TensorRT) · 8 chips · entry 6.1-0012 972 samples a second per chip

DLRM-v3

Offline · samples a second per chip

AMD Instinct MI355X AMD · smci355-ccs-aus-m13-13 (AS -4126GS-NMR-LCC) · 8 chips · entry 6.1-0003 1,628 samples a second per chip
AMD Instinct MI350X AMD · 8xMI350X_2xEPYC_9575F · 8 chips · entry 6.1-0001 1,361 samples a second per chip
AMD Instinct MI350P Dell_AMD · XE7745_8xMI350P_2xEPYC_9455 · 8 chips · entry 6.1-0040 605 samples a second per chip

Server · queries a second per chip

AMD Instinct MI355X AMD · smci355-ccs-aus-m13-13 (AS -4126GS-NMR-LCC) · 8 chips · entry 6.1-0003 1,525 queries a second per chip
AMD Instinct MI350X AMD · 8xMI350X_2xEPYC_9575F · 8 chips · entry 6.1-0001 1,162 queries a second per chip
AMD Instinct MI350P Dell_AMD · XE7745_8xMI350P_2xEPYC_9455 · 8 chips · entry 6.1-0040 563 queries a second per chip

Vision-language model (VLM)

Offline · samples a second per chip

NVIDIA GB300 Nebius · Nebius GB300 NVL72 (4x GB300-288GB_aarch64, TensorRT) · 4 chips · entry 6.1-0079 18.3 samples a second per chip
NVIDIA B300 GigaComputing · G894-ZD3-AAX7 · 8 chips · entry 6.1-0046 16.5 samples a second per chip
NVIDIA GB200 Crusoe · NVIDIA GB200 NVL72 (4x GB200-186GB_aarch64, Dynamo) · 4 chips · entry 6.1-0029 14.6 samples a second per chip
NVIDIA B200 Lambda · Lambda Cloud 8x B200-SXM-180GB · 8 chips · entry 6.1-0064 12.7 samples a second per chip

Server · queries a second per chip

NVIDIA GB300 NVIDIA · NVIDIA GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT) · 72 chips · entry 6.1-0074 16.8 queries a second per chip
NVIDIA B300 GigaComputing · G894-ZD3-AAX7 · 8 chips · entry 6.1-0046 14.5 queries a second per chip
NVIDIA GB200 RedHat_Supermicro · NVIDIA GB200 NVL4 (4x GB200-192GB_aarch64, vLLM) · 4 chips · entry 6.1-0093 12.7 queries a second per chip
NVIDIA B200 Lambda · Lambda Cloud 8x B200-SXM-180GB · 8 chips · entry 6.1-0064 8.68 queries a second per chip

Interactive · queries a second per chip

NVIDIA GB300 NVIDIA · NVIDIA GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT) · 72 chips · entry 6.1-0074 4.85 queries a second per chip
NVIDIA GB200 Crusoe · NVIDIA GB200 NVL72 (16x GB200-186GB_aarch64, Dynamo) · 16 chips · entry 6.1-0028 2.49 queries a second per chip
NVIDIA B300 GigaComputing · G894-ZD3-AAX7 · 8 chips · entry 6.1-0046 1.61 queries a second per chip

Text to video

Offline · samples a second per chip

NVIDIA B300 VibeHPC · NVIDIA DGX B300 (8x B300-SXM-270GB, TensorRT) · 8 chips · entry 6.1-0104 0.01 samples a second per chip
AMD Instinct MI355X AMD · smci355-ccs-aus-m13-13 (AS -4126GS-NMR-LCC) · 8 chips · entry 6.1-0003 0.01 samples a second per chip
NVIDIA GB300 NVIDIA · NVIDIA GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT) · 72 chips · entry 6.1-0074 0.01 samples a second per chip
AMD Instinct MI350X AMD · Quanta Grand Teton 1.5 MI350 · 8 chips · entry 6.1-0002 0.01 samples a second per chip
NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT) · 72 chips · entry 6.1-0072 0.01 samples a second per chip
AMD Instinct MI350P Dell_AMD · XE7745_8xMI350P_2xEPYC_9455 · 8 chips · entry 6.1-0040 0 samples a second per chip

End-to-end question answering

Offline · tasks a second per chip

Intel Arc Pro B70 Intel · 1-node-2S-Xeon_6787P · 4 chips · entry 6.1-0060 0.12 tasks a second per chip

End-to-end vector database

Offline · samples a second per chip

Intel Arc Pro B70 Intel · 1-node-2S-Xeon_6787P · 4 chips · entry 6.1-0060 5.65 samples a second per chip

Each chip’s best in MLPerf® Inference v6.0

published 1 Apr 2026 · 90 systems

Show the leaders per chip in v6.0

DeepSeek-R1

Offline · tokens a second per chip

NVIDIA GB300 CoreWeave · CoreWeave GB300 NVL72 (8x GB300-288GB_aarch64, TensorRT) · 8 chips · entry 6.0-0017 9,821 tokens a second per chip
NVIDIA B300 NVIDIA · NVIDIA DGX B300 (8x B300-SXM-270GB, TensorRT) · 8 chips · entry 6.0-0073 8,791 tokens a second per chip
NVIDIA GB200 CoreWeave · CoreWeave GB200 NVL72 (64x GB200-186GB_aarch64, TensorRT) · 64 chips · entry 6.0-0015 7,323 tokens a second per chip
NVIDIA B200 Nebius · Nebius B200 n1 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 6.0-0083 7,323 tokens a second per chip

Server · tokens a second per chip

NVIDIA GB300 Nebius · Nebius GB300 NVL72 (4x GB300-288GB_aarch64, TensorRT) · 8 chips · entry 6.0-0082 8,064 tokens a second per chip
NVIDIA B300 Nebius · Nebius B300 n1 (8x B300-SXM-270GB, TensorRT) · 8 chips · entry 6.0-0084 7,552 tokens a second per chip
NVIDIA B200 Nebius · Nebius B200 n1 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 6.0-0083 6,462 tokens a second per chip
NVIDIA GB200 CoreWeave · CoreWeave GB200 NVL72 (64x GB200-186GB_aarch64, TensorRT) · 64 chips · entry 6.0-0015 4,868 tokens a second per chip

Interactive · tokens a second per chip

NVIDIA GB300 NVIDIA · NVIDIA GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT) · 72 chips · entry 6.0-0078 3,481 tokens a second per chip
NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT) · 72 chips · entry 6.0-0075 3,338 tokens a second per chip
NVIDIA B300 GigaComputing · G894-SD3-AAX7 · 8 chips · entry 6.0-0039 617 tokens a second per chip

GPT-OSS-120B

Offline · tokens a second per chip

NVIDIA GB300 Nebius · Nebius GB300 NVL72 (4x GB300-288GB_aarch64, TensorRT) · 1 chip · entry 6.0-0079 15,372 tokens a second per chip
NVIDIA B300 Cisco · Cisco UCS C880A M8 (8x NVIDIA B300-SXM-270GB, TensorRT) · 8 chips · entry 6.0-0010 13,937 tokens a second per chip
NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT) · 72 chips · entry 6.0-0075 12,216 tokens a second per chip
AMD Instinct MI355X AMD · smci355-ccs-aus-m09-17 (AS -4126GS-NMR-LCC) · 8 chips · entry 6.0-0002 11,876 tokens a second per chip
NVIDIA B200 RedHat · LLM-D v0.5.0,Openshift 4.20.12,NVIDIA 8xB200-SXM-180GB · 8 chips · entry 6.0-0091 11,634 tokens a second per chip
NVIDIA H200 RedHat · LLM-D v0.5.0,OpenShift 4.21,IBM cloud gx3d-160x1792x8h200,NVIDIA 8xH200-SXM-141GB · 8 chips · entry 6.0-0092 3,585 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant Compute DL380a Gen12 (10x NVIDIA RTX PRO 6000 Blackwell Server Edition, TensorRT) · 10 chips · entry 6.0-0045 1,904 tokens a second per chip
Intel Arc Pro B60 Dual 48G Intel · 1-node-4x-BMG-Pro-B60-Dual · 4 chips · entry 6.0-0060 400 tokens a second per chip

Server · tokens a second per chip

NVIDIA GB300 Nebius · Nebius GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT) · 72 chips · entry 6.0-0081 15,233 tokens a second per chip
NVIDIA B300 Cisco · Cisco UCS C880A M8 (8x NVIDIA B300-SXM-270GB, TensorRT) · 8 chips · entry 6.0-0010 13,832 tokens a second per chip
NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT) · 72 chips · entry 6.0-0075 12,489 tokens a second per chip
NVIDIA B200 Nebius · Nebius B200 n1 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 6.0-0083 10,931 tokens a second per chip
AMD Instinct MI355X AMD · smci355-ccs-aus-m09-17 (AS -4126GS-NMR-LCC) · 8 chips · entry 6.0-0002 10,267 tokens a second per chip
NVIDIA H200 RedHat · LLM-D v0.5.0,OpenShift 4.21,IBM cloud gx3d-160x1792x8h200,NVIDIA 8xH200-SXM-141GB · 8 chips · entry 6.0-0092 3,013 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant Compute DL380a Gen12 (8x NVIDIA RTX PRO 6000 Blackwell Server Edition, TensorRT) · 8 chips · entry 6.0-0047 1,782 tokens a second per chip
Intel Arc Pro B60 Dual 48G Intel · 1-node-4x-BMG-Pro-B60-Dual · 4 chips · entry 6.0-0060 221 tokens a second per chip

Interactive · tokens a second per chip

NVIDIA GB300 NVIDIA · NVIDIA GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT) · 72 chips · entry 6.0-0078 9,406 tokens a second per chip
NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT) · 72 chips · entry 6.0-0075 8,680 tokens a second per chip
NVIDIA B300 ASUSTeK · XA NB3I-E12 · 8 chips · entry 6.0-0006 3,251 tokens a second per chip
NVIDIA B200 Nebius · Nebius B200 n1 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 6.0-0083 1,644 tokens a second per chip

Llama 2 70B, 99% accuracy

Offline · tokens a second per chip

NVIDIA GB300 NVIDIA · NVIDIA GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT) · 72 chips · entry 6.0-0078 15,651 tokens a second per chip
NVIDIA B300 Cisco · Cisco UCS C880A M8 (8x NVIDIA B300-SXM-270GB, TensorRT) · 8 chips · entry 6.0-0010 14,466 tokens a second per chip
NVIDIA B200 HPE · HPE ProLiant Compute XD685 (8x NVIDIA B200 180GB, TensorRT) · 8 chips · entry 6.0-0048 12,994 tokens a second per chip
AMD Instinct MI355X AMD · smci355-ccs-aus-m09-17 (AS -4126GS-NMR-LCC) · 8 chips · entry 6.0-0002 12,935 tokens a second per chip
NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT) · 72 chips · entry 6.0-0075 12,334 tokens a second per chip
AMD Instinct MI350X Supermicro_RedHat · AS-8126GS-TNMR,vLLM · 8 chips · entry 6.0-0100 10,060 tokens a second per chip
AMD Instinct MI325X MangoBoost · MangoBoost Mi325X (8x MI325X, LLMBoost) · 8 chips · entry 6.0-0070 4,235 tokens a second per chip
NVIDIA H200 NVL Dell · PowerEdge XE7740 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 6.0-0021 4,001 tokens a second per chip

Server · tokens a second per chip

NVIDIA B300 Cisco · Cisco UCS C880A M8 (8x NVIDIA B300-SXM-270GB, TensorRT) · 8 chips · entry 6.0-0010 14,313 tokens a second per chip
NVIDIA B200 HPE · HPE ProLiant Compute XD685 (8x NVIDIA B200 180GB, TensorRT) · 8 chips · entry 6.0-0048 12,954 tokens a second per chip
AMD Instinct MI355X AMD · smci355-ccs-aus-m09-17 (AS -4126GS-NMR-LCC) · 8 chips · entry 6.0-0002 12,535 tokens a second per chip
NVIDIA GB300 NVIDIA · NVIDIA GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT) · 72 chips · entry 6.0-0078 12,059 tokens a second per chip
NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT) · 72 chips · entry 6.0-0075 11,251 tokens a second per chip
AMD Instinct MI350X MiTAC · G8825Z5 · 8 chips · entry 6.0-0071 9,672 tokens a second per chip
AMD Instinct MI325X MangoBoost · MangoBoost Mi325X (8x MI325X, LLMBoost) · 8 chips · entry 6.0-0070 3,986 tokens a second per chip
NVIDIA H200 NVL Dell · PowerEdge XE7740 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 6.0-0021 3,636 tokens a second per chip

Interactive · tokens a second per chip

NVIDIA GB300 NVIDIA · NVIDIA GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT) · 72 chips · entry 6.0-0078 11,307 tokens a second per chip
NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT) · 72 chips · entry 6.0-0075 10,484 tokens a second per chip
AMD Instinct MI355X AMD · smci355-ccs-aus-m09-17 (AS -4126GS-NMR-LCC) · 8 chips · entry 6.0-0002 9,201 tokens a second per chip
NVIDIA B300 Dell · PowerEdge XE9780L (8x B300-SXM-270GB, TensorRT) · 8 chips · entry 6.0-0024 8,841 tokens a second per chip
NVIDIA B200 HPE · HPE ProLiant Compute XD685 (8x NVIDIA B200 180GB, TensorRT) · 8 chips · entry 6.0-0048 7,706 tokens a second per chip
AMD Instinct MI350X MiTAC · G8825Z5 · 8 chips · entry 6.0-0071 6,841 tokens a second per chip
NVIDIA H200 NVL Dell · PowerEdge XE7740 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 6.0-0021 2,043 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell ASUSTeK · ESC8000_E12P_RTX_PRO_6000_PCIE_96GBx8 · 8 chips · entry 6.0-0005 783 tokens a second per chip

Llama 2 70B, 99.9% accuracy

Offline · tokens a second per chip

NVIDIA GB300 NVIDIA · NVIDIA GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT) · 72 chips · entry 6.0-0078 15,651 tokens a second per chip
NVIDIA B300 NVIDIA · NVIDIA DGX B300 (8x B300-SXM-270GB, TensorRT) · 8 chips · entry 6.0-0073 14,119 tokens a second per chip
NVIDIA B200 HPE · HPE ProLiant Compute XD685 (8x NVIDIA B200 180GB, TensorRT) · 8 chips · entry 6.0-0048 13,072 tokens a second per chip
AMD Instinct MI355X AMD · smci355-ccs-aus-m09-17 (AS -4126GS-NMR-LCC) · 8 chips · entry 6.0-0002 12,935 tokens a second per chip
NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT) · 72 chips · entry 6.0-0075 12,334 tokens a second per chip
AMD Instinct MI350X MiTAC · G8825Z5 · 8 chips · entry 6.0-0071 9,874 tokens a second per chip
AMD Instinct MI325X MangoBoost · MangoBoost Mi325X (8x MI325X, LLMBoost) · 8 chips · entry 6.0-0070 4,235 tokens a second per chip
NVIDIA H200 NVL Dell · PowerEdge XE7740 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 6.0-0021 4,001 tokens a second per chip

Server · tokens a second per chip

NVIDIA B300 Dell · PowerEdge XE9780L (8x B300-SXM-270GB, TensorRT) · 8 chips · entry 6.0-0024 13,549 tokens a second per chip
NVIDIA B200 HPE · HPE ProLiant Compute XD685 (8x NVIDIA B200 180GB, TensorRT) · 8 chips · entry 6.0-0048 12,953 tokens a second per chip
AMD Instinct MI355X AMD · smci355-ccs-aus-m09-17 (AS -4126GS-NMR-LCC) · 8 chips · entry 6.0-0002 12,535 tokens a second per chip
NVIDIA GB300 NVIDIA · NVIDIA GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT) · 72 chips · entry 6.0-0078 12,059 tokens a second per chip
NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT) · 72 chips · entry 6.0-0075 11,251 tokens a second per chip
AMD Instinct MI350X MiTAC · G8825Z5 · 8 chips · entry 6.0-0071 9,672 tokens a second per chip
AMD Instinct MI325X MangoBoost · MangoBoost Mi325X (8x MI325X, LLMBoost) · 8 chips · entry 6.0-0070 3,986 tokens a second per chip
NVIDIA H200 NVL Dell · PowerEdge XE7740 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 6.0-0021 3,636 tokens a second per chip

Interactive · tokens a second per chip

NVIDIA GB300 NVIDIA · NVIDIA GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT) · 72 chips · entry 6.0-0078 11,307 tokens a second per chip
NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT) · 72 chips · entry 6.0-0075 10,484 tokens a second per chip
AMD Instinct MI355X AMD · smci355-ccs-aus-m09-17 (AS -4126GS-NMR-LCC) · 8 chips · entry 6.0-0002 9,201 tokens a second per chip
NVIDIA B300 Dell · PowerEdge XE9780L (8x B300-SXM-270GB, TensorRT) · 8 chips · entry 6.0-0024 8,841 tokens a second per chip
NVIDIA B200 HPE · HPE ProLiant Compute XD685 (8x NVIDIA B200 180GB, TensorRT) · 8 chips · entry 6.0-0048 7,663 tokens a second per chip
AMD Instinct MI350X MiTAC · G8825Z5 · 8 chips · entry 6.0-0071 6,841 tokens a second per chip
NVIDIA H200 NVL Dell · PowerEdge XE7740 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 6.0-0021 2,043 tokens a second per chip

Llama 3.1 405B

Offline · tokens a second per chip

NVIDIA GB300 NVIDIA · NVIDIA GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT) · 72 chips · entry 6.0-0078 271 tokens a second per chip
NVIDIA B300 Cisco · Cisco UCS C880A M8 (8x NVIDIA B300-SXM-270GB, TensorRT) · 8 chips · entry 6.0-0010 246 tokens a second per chip
NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT) · 72 chips · entry 6.0-0075 215 tokens a second per chip
NVIDIA B200 Google · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 6.0-0040 169 tokens a second per chip

Server · tokens a second per chip

NVIDIA GB300 NVIDIA · NVIDIA GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT) · 72 chips · entry 6.0-0078 259 tokens a second per chip
NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT) · 72 chips · entry 6.0-0075 196 tokens a second per chip
NVIDIA B300 Quanta_Cloud_Technology · QuantaGrid D75H-10U (8x B300-SXM-270GB, TensorRT) · 8 chips · entry 6.0-0089 186 tokens a second per chip
NVIDIA B200 Google · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 6.0-0040 123 tokens a second per chip

Interactive · tokens a second per chip

NVIDIA GB300 NVIDIA · NVIDIA GB300 NVL72 (72x GB300-288GB_aarch64, TensorRT) · 72 chips · entry 6.0-0078 255 tokens a second per chip
NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT) · 72 chips · entry 6.0-0075 195 tokens a second per chip
NVIDIA B300 GigaComputing · G894-SD3-AAX7 · 8 chips · entry 6.0-0039 95.6 tokens a second per chip

Llama 3.1 8B

Offline · tokens a second per chip

NVIDIA B300 ASUSTeK · XA NB3I-E12 · 8 chips · entry 6.0-0006 20,843 tokens a second per chip
NVIDIA B200 Lambda_SIT · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 6.0-0062 20,050 tokens a second per chip
NVIDIA H200 NVL Dell · PowerEdge XE7740 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 6.0-0021 6,649 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant DL385 Gen11 (2x NVIDIA RTX PRO 6000 Blackwell Server Edition-600W, TensorRT) · 2 chips · entry 6.0-0052 6,248 tokens a second per chip
NVIDIA RTX PRO 4500 Blackwell HPE · HPE ProLiant DL145 Gen11 (1x NVIDIA RTX PRO 4500 Blackwell, TensorRT) · 1 chip · entry 6.0-0043 2,341 tokens a second per chip
Intel Arc Pro B60 Dual 48G Intel · 1-node-4x-BMG-Pro-B60-Dual · 4 chips · entry 6.0-0060 1,691 tokens a second per chip
Intel Arc Pro B60 Intel · 1-node-4x-BMG-B60 · 4 chips · entry 6.0-0059 837 tokens a second per chip
Intel Arc Pro B50 Intel · 1-node-4x-BMG-B50 · 4 chips · entry 6.0-0058 430 tokens a second per chip

Server · tokens a second per chip

NVIDIA B300 ASUSTeK · XA NB3I-E12 · 8 chips · entry 6.0-0006 18,508 tokens a second per chip
NVIDIA B200 HPE · HPE ProLiant Compute XD685 (8x NVIDIA B200 180GB, TensorRT) · 8 chips · entry 6.0-0048 16,409 tokens a second per chip
NVIDIA H200 NVL Cisco · Cisco UCS X210 M8 - X580P PCIe Node (4x H200-NVL-141GB, TensorRT) · 4 chips · entry 6.0-0013 6,433 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant Compute DL380a Gen12 (8x NVIDIA RTX PRO 6000 Blackwell Server Edition, TensorRT) · 8 chips · entry 6.0-0047 6,099 tokens a second per chip
NVIDIA RTX PRO 4500 Blackwell HPE · HPE ProLiant Compute DL380 Gen12 (5x NVIDIA RTX PRO 4500 Blackwell, TensorRT) · 5 chips · entry 6.0-0050 2,270 tokens a second per chip
Intel Arc Pro B60 Dual 48G Intel · 1-node-4x-BMG-Pro-B60-Dual · 4 chips · entry 6.0-0060 1,573 tokens a second per chip
Intel Arc Pro B60 Intel · 1-node-4x-BMG-B60 · 4 chips · entry 6.0-0059 786 tokens a second per chip
Intel Arc Pro B50 Intel · 1-node-4x-BMG-B50 · 4 chips · entry 6.0-0058 297 tokens a second per chip

Interactive · tokens a second per chip

NVIDIA B200 Lambda_SIT · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 6.0-0062 16,094 tokens a second per chip
NVIDIA B300 GigaComputing · G894-SD3-AAX7 · 8 chips · entry 6.0-0039 16,079 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant Compute DL380a Gen12 (8x NVIDIA RTX PRO 6000 Blackwell Server Edition, TensorRT) · 8 chips · entry 6.0-0047 5,511 tokens a second per chip

Mixtral 8x7B

Offline · tokens a second per chip

NVIDIA B200 HPE · HPE ProLiant Compute XD685 (8x NVIDIA B200 180GB, TensorRT) · 8 chips · entry 6.0-0048 15,809 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant Compute DL380a Gen12 (8x NVIDIA RTX PRO 6000 Blackwell Server Edition, TensorRT) · 8 chips · entry 6.0-0046 4,205 tokens a second per chip

Server · tokens a second per chip

NVIDIA B200 HPE · HPE ProLiant Compute XD685 (8x NVIDIA B200 180GB, TensorRT) · 8 chips · entry 6.0-0048 15,901 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant Compute DL380a Gen12 (8x NVIDIA RTX PRO 6000 Blackwell Server Edition, TensorRT) · 8 chips · entry 6.0-0046 4,019 tokens a second per chip

Whisper

Offline · samples a second per chip

NVIDIA B300 Supermicro · NVIDIA DGX B300 (8x B300-SXM-270GB, TensorRT) · 8 chips · entry 6.0-0098 6,320 samples a second per chip
NVIDIA B200 Google · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 6.0-0040 6,166 samples a second per chip
NVIDIA GB200 Google · NVIDIA GB200 NVL72 (4x GB200-186GB_aarch64, TensorRT) · 4 chips · entry 6.0-0041 5,513 samples a second per chip
NVIDIA H200 RedHat · IBM Cloud qx3d-160x1792x8h200,NVIDIA 8xH200-SXM-141GB,vLLM 0.15.1 · 8 chips · entry 6.0-0093 4,549 samples a second per chip
NVIDIA H200 NVL Dell · PowerEdge XE7740 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 6.0-0021 3,996 samples a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant DL385 Gen11 (2x NVIDIA RTX PRO 6000 Blackwell Server Edition-600W, TensorRT) · 2 chips · entry 6.0-0052 2,356 samples a second per chip
NVIDIA L40S RedHat · IBM Cloud gx3-48x240x2l40s instance,2xL40S-PCIE-48GB,vLLM 0.15.1 · 2 chips · entry 6.0-0090 1,823 samples a second per chip
Intel Arc Pro B60 Intel · 1-node-4x-BMG-B60 · 4 chips · entry 6.0-0059 999 samples a second per chip

DLRM-v3

Offline · samples a second per chip

NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT) · 72 chips · entry 6.0-0075 1,453 samples a second per chip
NVIDIA B200 Netweb_technologies · Camarero PDI200A2HG-810 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 6.0-0085 1,342 samples a second per chip

Server · queries a second per chip

NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (72x GB200-186GB_aarch64, TensorRT) · 72 chips · entry 6.0-0075 1,389 queries a second per chip
NVIDIA B200 Google · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 6.0-0040 1,251 queries a second per chip

Qwen3-VL-235B-A22B

Offline · samples a second per chip

NVIDIA GB300 NVIDIA · NVIDIA GB300 NVL72 (4x GB300-288GB_aarch64, TensorRT) · 4 chips · entry 6.0-0077 15.3 samples a second per chip
NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (4x GB200-186GB_aarch64, TensorRT) · 4 chips · entry 6.0-0074 10.9 samples a second per chip
NVIDIA B200 RedHat · Dell B200,8xB200-SXM-180GB,RHEL 10.1,vLLM CentML:mlperf-inf-mm-q3vl-v6.0 · 8 chips · entry 6.0-0094 9.88 samples a second per chip
NVIDIA B300 Nebius · Nebius B300 n1 (8x B300-SXM-270GB, TensorRT) · 8 chips · entry 6.0-0084 9.78 samples a second per chip
NVIDIA H200 RedHat · IBM Cloud gx3d-160x1792x8h200,8xH200-SXM-141GB,RHEL 9.6,vLLM CentML:mlperf-inf-mm-q3vl-v6.0 · 8 chips · entry 6.0-0095 2.28 samples a second per chip

Server · queries a second per chip

NVIDIA GB300 Nebius · Nebius GB300 NVL72 (4x GB300-288GB_aarch64, TensorRT) · 4 chips · entry 6.0-0080 10.8 queries a second per chip
NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (4x GB200-186GB_aarch64, TensorRT) · 4 chips · entry 6.0-0074 9.51 queries a second per chip
NVIDIA B200 RedHat · Dell B200,8xB200-SXM-180GB,RHEL 10.1,vLLM CentML:mlperf-inf-mm-q3vl-v6.0 · 8 chips · entry 6.0-0094 8.48 queries a second per chip
NVIDIA B300 Nebius · Nebius B300 n1 (8x B300-SXM-270GB, TensorRT) · 8 chips · entry 6.0-0084 5.64 queries a second per chip
NVIDIA H200 RedHat · IBM Cloud gx3d-160x1792x8h200,8xH200-SXM-141GB,RHEL 9.6,vLLM CentML:mlperf-inf-mm-q3vl-v6.0 · 8 chips · entry 6.0-0095 1.38 queries a second per chip

Wan 2.2 text to video

Offline · samples a second per chip

NVIDIA GB300 NVIDIA · NVIDIA GB300 NVL72 (4x GB300-288GB_aarch64, TensorRT) · 4 chips · entry 6.0-0077 0.01 samples a second per chip
NVIDIA B300 NVIDIA · NVIDIA DGX B300 (8x B300-SXM-270GB, TensorRT) · 8 chips · entry 6.0-0073 0.01 samples a second per chip
NVIDIA GB200 NVIDIA · NVIDIA GB200 NVL72 (4x GB200-186GB_aarch64, TensorRT) · 4 chips · entry 6.0-0074 0.01 samples a second per chip
NVIDIA B200 NVIDIA · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 6.0-0072 0.01 samples a second per chip
NVIDIA H200 Krai · HPE Cray XD670 (8x H200-SXM-141GB, KISS-V) · 8 chips · entry 6.0-0061 0 samples a second per chip

Each chip’s best in MLPerf® Inference v5.1

published 9 Sept 2025 · 82 systems

Show the leaders per chip in v5.1

DeepSeek-R1

Offline · tokens a second per chip

NVIDIA GB300 NVIDIA · 72x GB300-288GB_aarch64, TensorRT · 72 chips · entry 5.1-0072 5,842 tokens a second per chip
NVIDIA B200 NVIDIA · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 5.1-0069 4,172 tokens a second per chip
NVIDIA GB200 NVIDIA · 72x GB200-186GB_aarch64, TensorRT · 72 chips · entry 5.1-0071 4,024 tokens a second per chip

Server · tokens a second per chip

NVIDIA GB300 NVIDIA · 72x GB300-288GB_aarch64, TensorRT · 72 chips · entry 5.1-0072 2,907 tokens a second per chip
NVIDIA GB200 NVIDIA · 72x GB200-186GB_aarch64, TensorRT · 72 chips · entry 5.1-0071 2,327 tokens a second per chip
NVIDIA B200 NVIDIA · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 5.1-0069 2,324 tokens a second per chip

Llama 2 70B, 99% accuracy

Offline · tokens a second per chip

NVIDIA GB200 Azure · ND_GB200_v6 · 4 chips · entry 5.1-0008 13,015 tokens a second per chip
NVIDIA B200 Lenovo · ThinkSystem SR680a V3 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 5.1-0061 12,864 tokens a second per chip
NVIDIA H200 Dell · Dell PowerEdge XE9680 (8xH200-SXM-141GB, TensorRT) · 8 chips · entry 5.1-0026 4,415 tokens a second per chip
AMD Instinct MI325X AMD · QuantaGrid D74A-7U · 8 chips · entry 5.1-0003 4,319 tokens a second per chip
NVIDIA H200 NVL Dell · Dell PowerEdge XE7745 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 5.1-0023 3,908 tokens a second per chip
NVIDIA H100 Cisco · HPF HGX System (32xH100-SXM-80GB, TensorRT) · 32 chips · entry 5.1-0016 3,902 tokens a second per chip
AMD Instinct MI300X MangoBoost · Supermicro AS-8125GS-TNMR2 (8x MI300X-192GB MangoBoost-LLMBoost) · 8 chips · entry 5.1-0066 3,482 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant Compute DL380a Gen12 (8x RTX Pro 6000 Blackwell Server Edition) · 8 chips · entry 5.1-0051 3,276 tokens a second per chip

Server · tokens a second per chip

NVIDIA B200 Nebius · Nebius B200 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 5.1-0073 12,701 tokens a second per chip
NVIDIA GB200 NVIDIA · 4x GB200-186GB_aarch64, TensorRT · 4 chips · entry 5.1-0070 12,340 tokens a second per chip
NVIDIA H200 ASUSTeK · ASUSTeK ESC N8 H200 (8x H200-SXM-141GB, TensorRT) · 8 chips · entry 5.1-0007 4,274 tokens a second per chip
AMD Instinct MI325X GigaComputing · G893-ZX1-AAX2 · 8 chips · entry 5.1-0045 4,017 tokens a second per chip
NVIDIA H100 Cisco · HPF HGX System (32xH100-SXM-80GB, TensorRT) · 32 chips · entry 5.1-0016 3,821 tokens a second per chip
NVIDIA H200 NVL Dell · Dell PowerEdge XE7745 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 5.1-0023 3,634 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant Compute DL380a Gen12 (8x RTX Pro 6000 Blackwell Server Edition) · 8 chips · entry 5.1-0051 3,251 tokens a second per chip
AMD Instinct MI300X MangoBoost · Supermicro AS-8125GS-TNMR2 (8x MI300X-192GB MangoBoost-LLMBoost) · 8 chips · entry 5.1-0066 3,108 tokens a second per chip

Interactive · tokens a second per chip

NVIDIA B200 GigaComputing · G894-SD1 · 8 chips · entry 5.1-0046 7,856 tokens a second per chip
NVIDIA GB200 NVIDIA · 4x GB200-186GB_aarch64, TensorRT · 4 chips · entry 5.1-0070 7,436 tokens a second per chip
NVIDIA H200 Nebius · Nebius H200 (8x H200-SXM-141GB, TensorRT) · 8 chips · entry 5.1-0075 2,885 tokens a second per chip
AMD Instinct MI325X MiTAC · G8825Z5 · 8 chips · entry 5.1-0068 2,356 tokens a second per chip
NVIDIA H200 NVL Dell · Dell PowerEdge XE7745 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 5.1-0023 2,052 tokens a second per chip
AMD Instinct MI300X AMD · Supermicro AS-8125GS-TNMR2 · 8 chips · entry 5.1-0001 1,105 tokens a second per chip

Llama 2 70B, 99.9% accuracy

Offline · tokens a second per chip

NVIDIA GB200 NVIDIA · 4x GB200-186GB_aarch64, TensorRT · 4 chips · entry 5.1-0070 12,934 tokens a second per chip
NVIDIA B200 Lenovo · ThinkSystem SR680a V3 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 5.1-0061 12,864 tokens a second per chip
NVIDIA H200 Dell · Dell PowerEdge XE9680 (8xH200-SXM-141GB, TensorRT) · 8 chips · entry 5.1-0026 4,415 tokens a second per chip
AMD Instinct MI325X AMD · QuantaGrid D74A-7U · 8 chips · entry 5.1-0003 4,319 tokens a second per chip
NVIDIA H200 NVL Dell · Dell PowerEdge XE7745 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 5.1-0023 3,908 tokens a second per chip
NVIDIA H100 Cisco · HPF HGX System (32xH100-SXM-80GB, TensorRT) · 32 chips · entry 5.1-0016 3,902 tokens a second per chip
AMD Instinct MI300X MangoBoost · Supermicro AS-8125GS-TNMR2 (8x MI300X-192GB MangoBoost-LLMBoost) · 8 chips · entry 5.1-0066 3,482 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant Compute DL380a Gen12 (8x RTX Pro 6000 Blackwell Server Edition) · 8 chips · entry 5.1-0051 3,276 tokens a second per chip

Server · tokens a second per chip

NVIDIA B200 Nebius · Nebius B200 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 5.1-0073 12,701 tokens a second per chip
NVIDIA GB200 NVIDIA · 4x GB200-186GB_aarch64, TensorRT · 4 chips · entry 5.1-0070 12,340 tokens a second per chip
NVIDIA H200 Nebius · Nebius H200 (8x H200-SXM-141GB, TensorRT) · 8 chips · entry 5.1-0075 4,254 tokens a second per chip
AMD Instinct MI325X GigaComputing · G893-ZX1-AAX2 · 8 chips · entry 5.1-0045 4,017 tokens a second per chip
NVIDIA H100 Cisco · HPF HGX System (32xH100-SXM-80GB, TensorRT) · 32 chips · entry 5.1-0016 3,821 tokens a second per chip
NVIDIA H200 NVL Dell · Dell PowerEdge XE7745 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 5.1-0023 3,634 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant Compute DL380a Gen12 (8x RTX Pro 6000 Blackwell Server Edition) · 8 chips · entry 5.1-0051 3,250 tokens a second per chip
AMD Instinct MI300X MangoBoost · Supermicro AS-8125GS-TNMR2 (8x MI300X-192GB MangoBoost-LLMBoost) · 8 chips · entry 5.1-0066 3,108 tokens a second per chip

Interactive · tokens a second per chip

NVIDIA B200 GigaComputing · G894-SD1 · 8 chips · entry 5.1-0046 7,856 tokens a second per chip
NVIDIA GB200 NVIDIA · 4x GB200-186GB_aarch64, TensorRT · 4 chips · entry 5.1-0070 7,436 tokens a second per chip
NVIDIA H200 Nebius · Nebius H200 (8x H200-SXM-141GB, TensorRT) · 8 chips · entry 5.1-0075 2,885 tokens a second per chip
AMD Instinct MI325X MiTAC · G8825Z5 · 8 chips · entry 5.1-0068 2,356 tokens a second per chip
NVIDIA H200 NVL Dell · Dell PowerEdge XE7745 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 5.1-0023 2,052 tokens a second per chip
AMD Instinct MI300X AMD · Supermicro AS-8125GS-TNMR2 · 8 chips · entry 5.1-0001 1,105 tokens a second per chip

Llama 3.1 405B

Offline · tokens a second per chip

NVIDIA GB300 NVIDIA · 72x GB300-288GB_aarch64, TensorRT · 72 chips · entry 5.1-0072 224 tokens a second per chip
NVIDIA GB200 Nebius · Nebius GB200 (4x GB200-186GB_aarch64, TensorRT) · 4 chips · entry 5.1-0074 214 tokens a second per chip
NVIDIA B200 Dell · Dell PowerEdge XE9685L (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 5.1-0028 208 tokens a second per chip
NVIDIA H200 Nebius · Nebius H200 (8x H200-SXM-141GB, TensorRT) · 8 chips · entry 5.1-0075 69.1 tokens a second per chip
NVIDIA H100 Cisco · HPF HGX System (16xH100-SXM-80GB, TensorRT) · 16 chips · entry 5.1-0015 48.9 tokens a second per chip

Server · tokens a second per chip

NVIDIA GB300 NVIDIA · 72x GB300-288GB_aarch64, TensorRT · 72 chips · entry 5.1-0072 170 tokens a second per chip
NVIDIA GB200 NVIDIA · 72x GB200-186GB_aarch64, TensorRT · 72 chips · entry 5.1-0071 161 tokens a second per chip
NVIDIA B200 Nebius · Nebius B200 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 5.1-0073 160 tokens a second per chip
NVIDIA H200 Quanta_Cloud_Technology · QuantaGrid D74H-7U (8x H200-SXM-141GB, TensorRT) · 8 chips · entry 5.1-0079 36.9 tokens a second per chip
NVIDIA H100 Cisco · HPF HGX System (32xH100-SXM-80GB, TensorRT) · 32 chips · entry 5.1-0016 35.2 tokens a second per chip

Interactive · tokens a second per chip

NVIDIA GB200 NVIDIA · 72x GB200-186GB_aarch64, TensorRT · 72 chips · entry 5.1-0071 138 tokens a second per chip
NVIDIA B200 Nebius · Nebius B200 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 5.1-0073 96.3 tokens a second per chip
NVIDIA H200 Nebius · Nebius H200 (8x H200-SXM-141GB, TensorRT) · 8 chips · entry 5.1-0075 25.3 tokens a second per chip

Llama 3.1 8B

Offline · tokens a second per chip

NVIDIA B200 Lenovo · ThinkSystem SR780a V3 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 5.1-0062 18,370 tokens a second per chip
NVIDIA H200 HPE · HPE Cray XD670 (8x H200-SXM-141GB) · 8 chips · entry 5.1-0049 8,255 tokens a second per chip
NVIDIA H200 NVL Cisco · Cisco UCS C845A M8 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 5.1-0011 6,347 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant Compute DL380a Gen12 (8x RTX Pro 6000 Blackwell Server Edition) · 8 chips · entry 5.1-0051 5,855 tokens a second per chip
NVIDIA H100 RedHat · Dell PowerEdge XE8640 (1xH100-SXM-80GB, vLLM) · 1 chip · entry 5.1-0083 5,777 tokens a second per chip
NVIDIA L40S RedHat · IBM Cloud gx3 instance 1xL40S-PCIE-48GB ,vLLM · 1 chip · entry 5.1-0082 1,642 tokens a second per chip
Intel Arc Pro B60 Dual 48G Intel · 1-node-4x-BMG-Pro-B60-Dual · 4 chips · entry 5.1-0056 1,618 tokens a second per chip

Server · tokens a second per chip

NVIDIA B200 Dell · Dell PowerEdge XE9685L (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 5.1-0028 16,099 tokens a second per chip
NVIDIA H200 HPE · HPE Cray XD670 (8x H200-SXM-141GB) · 8 chips · entry 5.1-0049 8,114 tokens a second per chip
NVIDIA H200 NVL Quanta_Cloud_Technology · D75E-4U_H200-NVL-141GBx4 · 4 chips · entry 5.1-0080 6,438 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant Compute DL380a Gen12 (8x RTX Pro 6000 Blackwell Server Edition) · 8 chips · entry 5.1-0051 5,758 tokens a second per chip
NVIDIA H100 RedHat · Dell PowerEdge XE8640 (1xH100-SXM-80GB, vLLM) · 1 chip · entry 5.1-0083 5,104 tokens a second per chip
Intel Arc Pro B60 Dual 48G Intel · 1-node-4x-BMG-Pro-B60-Dual · 4 chips · entry 5.1-0056 1,337 tokens a second per chip
NVIDIA L40S RedHat · IBM Cloud gx3 instance 1xL40S-PCIE-48GB ,vLLM · 1 chip · entry 5.1-0082 1,207 tokens a second per chip

Interactive · queries a second per chip

NVIDIA B200 Supermicro · AS-4126GS-NBR-LCC (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 5.1-0086 15,284 queries a second per chip
NVIDIA H200 Quanta_Cloud_Technology · QuantaGrid D74H-7U (8x H200-SXM-141GB, TensorRT) · 8 chips · entry 5.1-0079 6,765 queries a second per chip
NVIDIA H200 NVL Quanta_Cloud_Technology · D75E-4U_H200-NVL-141GBx4 · 4 chips · entry 5.1-0080 5,486 queries a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant Compute DL380a Gen12 (8x RTX Pro 6000 Blackwell Server Edition) · 8 chips · entry 5.1-0051 4,507 queries a second per chip

Mixtral 8x7B

Offline · tokens a second per chip

AMD Instinct MI325X Vultr · Supermicro AS -8126GS-TNMR (8x MI325X) · 8 chips · entry 5.1-0096 8,754 tokens a second per chip
NVIDIA H200 HPE · HPE Cray XD670 (8x H200-SXM-141GB) · 8 chips · entry 5.1-0049 7,764 tokens a second per chip
AMD Instinct MI300X Dell · Dell Poweredge XE9680 · 8 chips · entry 5.1-0027 6,667 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant Compute DL380a Gen12 (8x RTX Pro 6000 Blackwell Server Edition) · 8 chips · entry 5.1-0051 4,092 tokens a second per chip

Server · tokens a second per chip

NVIDIA H200 HPE · HPE Cray XD670 (8x H200-SXM-141GB) · 8 chips · entry 5.1-0049 7,619 tokens a second per chip
AMD Instinct MI325X AMD · QuantaGrid D74A-7U · 8 chips · entry 5.1-0002 7,473 tokens a second per chip
AMD Instinct MI300X AMD · Supermicro AS-8125GS-TNMR2 · 8 chips · entry 5.1-0001 5,976 tokens a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant Compute DL380a Gen12 (8x RTX Pro 6000 Blackwell Server Edition) · 8 chips · entry 5.1-0051 3,767 tokens a second per chip

Whisper

Offline · samples a second per chip

NVIDIA B200 NVIDIA · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 5.1-0069 5,667 samples a second per chip
NVIDIA GB200 Oracle · BM.GPU.GB200.4 · 4 chips · entry 5.1-0077 5,568 samples a second per chip
NVIDIA H200 HPE · HPE Cray XD670 (8x H200-SXM-141GB) · 8 chips · entry 5.1-0049 4,306 samples a second per chip
NVIDIA H200 NVL HPE · HPE ProLiant DL385 Gen11 (2x H200-NVL-141GB) · 2 chips · entry 5.1-0052 3,963 samples a second per chip
NVIDIA H100 NVL Dell · Dell PowerEdge R7715 (3x H100-NVL-94GB, TensorRT) · 3 chips · entry 5.1-0021 2,767 samples a second per chip
NVIDIA L40S Dell · Dell PowerEdge XE7740 (8x L40S, TensorRT) · 8 chips · entry 5.1-0022 1,159 samples a second per chip
NVIDIA L4 Dell · PowerEdge R570 (4x L4-PCIE-24GBx4, TensorRT) · 4 chips · entry 5.1-0019 349 samples a second per chip
NVIDIA GeForce RTX 4090 GATEOverflow · GATE Overflow Intel Sapphire Rapids (2x RTX 4090) · 2 chips · entry 5.1-0033 52.5 samples a second per chip

R-GAT

Offline · samples a second per chip

NVIDIA B200 NVIDIA · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 5.1-0069 81,404 samples a second per chip

RetinaNet

Offline · samples a second per chip

NVIDIA H200 HPE · HPE Cray XD670 (8x H200-SXM-141GB) · 8 chips · entry 5.1-0049 1,875 samples a second per chip
NVIDIA H200 NVL Dell · Dell PowerEdge XE7745 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 5.1-0023 1,678 samples a second per chip
NVIDIA GeForce RTX 4090 GATEOverflow · GATE Overflow Intel Sapphire Rapids (2x RTX 4090) · 2 chips · entry 5.1-0032 880 samples a second per chip
NVIDIA L40S Cisco · Cisco UCS C845A M8 (8x L40S-48GB, TensorRT) · 8 chips · entry 5.1-0012 815 samples a second per chip
NVIDIA L4 Dell · PowerEdge R470 (2x L4-PCIE-24GB, TensorRT) · 2 chips · entry 5.1-0018 229 samples a second per chip

Server · queries a second per chip

NVIDIA H200 ASUSTeK · ASUSTeK ESC N8 H200 (8x H200-SXM-141GB, TensorRT) · 8 chips · entry 5.1-0007 1,801 queries a second per chip
NVIDIA H200 NVL Cisco · Cisco UCS C845A M8 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 5.1-0011 1,650 queries a second per chip
NVIDIA L40S Cisco · Cisco UCS C845A M8 (8x L40S-48GB, TensorRT) · 8 chips · entry 5.1-0012 763 queries a second per chip
NVIDIA GeForce RTX 4090 GATEOverflow · GATE Overflow Intel Sapphire Rapids (2x RTX 4090) · 2 chips · entry 5.1-0032 724 queries a second per chip
NVIDIA L4 Dell · PowerEdge R470 (2x L4-PCIE-24GB, TensorRT) · 2 chips · entry 5.1-0018 214 queries a second per chip

Stable Diffusion XL

Offline · samples a second per chip

NVIDIA B200 Lambda · NVIDIA DGX B200 (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 5.1-0060 4.07 samples a second per chip
NVIDIA H200 Quanta_Cloud_Technology · QuantaGrid D74H-7U (8x H200-SXM-141GB, TensorRT) · 8 chips · entry 5.1-0079 2.4 samples a second per chip
AMD Instinct MI325X AMD · Quanta S7PA · 8 chips · entry 5.1-0004 2.32 samples a second per chip
NVIDIA H100 The_Stage · Nebius H100 (8x H100-SXM-80GB, TheStageAI) · 8 chips · entry 5.1-0092 2.26 samples a second per chip
NVIDIA H200 NVL Dell · Dell PowerEdge XE7745 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 5.1-0023 2.14 samples a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant Compute DL380a Gen12 (8x RTX Pro 6000 Blackwell Server Edition) · 8 chips · entry 5.1-0051 1.38 samples a second per chip
NVIDIA L40S Dell · Dell PowerEdge XE7740 (8x L40S, TensorRT) · 8 chips · entry 5.1-0022 0.69 samples a second per chip
NVIDIA L4 Dell · PowerEdge R570 (4x L4-PCIE-24GBx4, TensorRT) · 4 chips · entry 5.1-0019 0.25 samples a second per chip

Server · queries a second per chip

NVIDIA B200 Broadcom_Supermicro · Supermicro SYS-422GA-NBRT-LCC (8x B200-SXM-180GB, TensorRT) · 8 chips · entry 5.1-0009 3.59 queries a second per chip
NVIDIA H200 Quanta_Cloud_Technology · QuantaGrid D74H-7U (8x H200-SXM-141GB, TensorRT) · 8 chips · entry 5.1-0079 2.3 queries a second per chip
NVIDIA H100 The_Stage · Nebius H100 (8x H100-SXM-80GB, TheStageAI) · 8 chips · entry 5.1-0092 2.23 queries a second per chip
NVIDIA H200 NVL Dell · Dell PowerEdge XE7745 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 5.1-0023 2.03 queries a second per chip
AMD Instinct MI325X AMD · Quanta S7PA · 8 chips · entry 5.1-0004 2.03 queries a second per chip
NVIDIA RTX PRO 6000 Blackwell HPE · HPE ProLiant Compute DL380a Gen12 (8x RTX Pro 6000 Blackwell Server Edition) · 8 chips · entry 5.1-0051 1.36 queries a second per chip
NVIDIA L40S Dell · Dell PowerEdge XE7740 (8x L40S, TensorRT) · 8 chips · entry 5.1-0022 0.74 queries a second per chip
NVIDIA L4 Dell · PowerEdge R570 (4x L4-PCIE-24GBx4, TensorRT) · 4 chips · entry 5.1-0019 0.22 queries a second per chip

DLRM-v2, 99% accuracy

Offline · samples a second per chip

NVIDIA GH200 HPE · HPE ProLiant Compute DL384 Gen12 (2x GH200-144GB_aarch64) · 2 chips · entry 5.1-0053 87,228 samples a second per chip
NVIDIA H200 Quanta_Cloud_Technology · QuantaGrid D74H-7U (8x H200-SXM-141GB, TensorRT) · 8 chips · entry 5.1-0079 80,983 samples a second per chip
NVIDIA H200 NVL ASUSTeK · ESC8000A-E12 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 5.1-0005 72,558 samples a second per chip

Server · queries a second per chip

NVIDIA GH200 HPE · HPE ProLiant Compute DL384 Gen12 (2x GH200-144GB_aarch64) · 2 chips · entry 5.1-0053 80,515 queries a second per chip
NVIDIA H200 ASUSTeK · ASUSTeK ESC N8 H200 (8x H200-SXM-141GB, TensorRT) · 8 chips · entry 5.1-0007 73,895 queries a second per chip
NVIDIA H200 NVL HPE · HPE ProLiant Compute DL380a Gen12 (10x H200-NVL-141GB) · 10 chips · entry 5.1-0050 65,021 queries a second per chip

DLRM-v2, 99.9% accuracy

Offline · samples a second per chip

NVIDIA H200 Quanta_Cloud_Technology · QuantaGrid D74H-7U (8x H200-SXM-141GB, TensorRT) · 8 chips · entry 5.1-0079 48,666 samples a second per chip
NVIDIA H200 NVL ASUSTeK · ESC8000A-E12 (8x H200-NVL-141GB, TensorRT) · 8 chips · entry 5.1-0005 43,528 samples a second per chip

Server · queries a second per chip

NVIDIA H200 Quanta_Cloud_Technology · QuantaGrid D74H-7U (8x H200-SXM-141GB, TensorRT) · 8 chips · entry 5.1-0079 46,259 queries a second per chip
NVIDIA H200 NVL HPE · HPE ProLiant Compute DL380a Gen12 (10x H200-NVL-141GB) · 10 chips · entry 5.1-0050 41,357 queries a second per chip

The rounds on this page

Datacenter suite, Closed division, Available category

Each round of MLPerf® Inference: Datacenter this page shows
Round	Published	Systems	Results	Source
MLPerf® Inference v6.1	16 Sept 2026	95	447	mlcommons/inference_results_v6.1commit 4bb63cd
MLPerf® Inference v6.0	1 Apr 2026	90	442	mlcommons/inference_results_v6.0commit 4d3916a
MLPerf® Inference v5.1	9 Sept 2025	82	552	mlcommons/inference_results_v5.1commit 5ea4f62

v5.0: Not shown. The v5.0 results repository, at commit 0bc17ab3f4b33f730d7869e04cc29cb70835e513, holds no licence file, and this site shows no data without a licence. v4.1 and before: Not shown. The results repositories before v5.0 have no summary file at their root, only each submitter's logs. Results in the Preview and research categories are left out of each round: 10 in v6.1, 9 in v6.0, 2 in v5.1.

What a figure per chip is

and the results that have none

Each MLPerf® Inference result is a whole system’s throughput: tokens, samples or queries a second, in one scenario of one benchmark. This page divides it by the number of accelerators the summary states for the system. A system with more than one kind of chip, or none, gets no figure per chip: 30 systems across the rounds. A latency result gets none either. MLCommons verifies each system result, not a figure per chip.

Where a summary states the count of chips two ways that disagree, the page gives no figure per chip: entry 6.0-0001 (AMD), where the summary states 88 in all, 11 nodes of 8, x87 in the name; entry 6.0-0003 (AMD), where the summary states 96 in all, 12 nodes of 8, x94 in the name.

Sources

inference.csv inference-gains.csv Every dataset → Status → Glossary →

MLCommons: MLPerf® Inference v5.1 results summary, at commit 5ea4f62 of mlcommons/inference_results_v5.1, under the Apache License 2.0; retrieved 10 Nov 2026 and kept with sha256 68c19b46cf5a. Results verified by MLCommons Association. Each row of inference.csv names its entry.
MLCommons: MLPerf® Inference v6.0 results summary, at commit 4d3916a of mlcommons/inference_results_v6.0, under the Apache License 2.0; retrieved 10 Nov 2026 and kept with sha256 52fa813a2783. Results verified by MLCommons Association. Each row of inference.csv names its entry.
MLCommons: MLPerf® Inference v6.1 results summary, at commit 4bb63cd of mlcommons/inference_results_v6.1, under the Apache License 2.0; retrieved 10 Nov 2026 and kept with sha256 226adca4c8cf. Results verified by MLCommons Association. Each row of inference.csv names its entry.
MLCommons: the MLPerf® compatibility table and the MLPerf® results messaging guidelines.
The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.