Chips · inference benchmark

NVIDIA B200 served 2.8 times as many DeepSeek-R1 tokens per chip

Server scenario: the best NVIDIA B200 result per chip went from 2,324 to 6,462 tokens a second between MLPerf® Inference v5.1 and v6.0. A figure per chip is this site’s division of a result by the system’s count of accelerators. It is not a primary metric of MLPerf® Inference, and it is not verified by MLCommons Association. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

NVIDIA B200 DeepSeek-R1 · Server · v5.1 to v6.0 · 5.1-0069 and 6.0-0083 2.8× times the result per chip of the round before
NVIDIA GB300 DeepSeek-R1 · Server · v5.1 to v6.0 · 5.1-0072 and 6.0-0082 2.8× times the result per chip of the round before
NVIDIA GB200 DeepSeek-R1 · Server · v5.1 to v6.0 · 5.1-0071 and 6.0-0015 2.1× times the result per chip of the round before
NVIDIA GB200 DeepSeek-R1 · Offline · v5.1 to v6.0 · 5.1-0071 and 6.0-0015 1.8× times the result per chip of the round before