Chips · accelerators · NVIDIA

NVIDIA Tesla V100 SXM2 · Volta

417 gigaflops of dense FP16 compute per watt, at 300 W

Shipped 10 Aug 2017; shipped by the first date promised. Epoch AI names it at no AI data center and 46 GPU clusters.

Embed

The frame credits NVIDIA, Epoch AI, AI data centers and Epoch AI, Data on GPU clusters and links back to its page. Preview it.

Cite
CSV Post on X

Each figure in NVIDIA’s own words

one chip, dense compute; memory is high-bandwidth memory
FP16, dense
125 TFLOPS Tesla V100 delivers 125 teraFLOPS of deep learning performance NVIDIA Tesla V100 GPU accelerator datasheet (Mar 2018)
Memory
32 GB GPU Memory 32GB /16GB HBM2 NVIDIA Tesla V100 GPU accelerator datasheet (Mar 2018)
Memory bandwidth
0.9 TB/s Memory Bandwidth 900GB/sec NVIDIA Tesla V100 GPU accelerator datasheet (Mar 2018) (900 GB/s on the page)
Maximum power
300 W PCIe Full Height/Length SXM2 250 W 300 W Max Power Comsumption NVIDIA Tesla V100 GPU accelerator datasheet (Mar 2018) (read from a table with a column for each product, whose head reads “PCIe Full Height/Length SXM2 250 W 300 W Max Power Comsumption”)
Compute per watt
417 GFLOPS/Wdense FP16 over maximum power

V100 has no BF16 or FP8, so fp16 gives the Tensor Core figure (125 TFLOPS for SXM2, from the datasheet text). The datasheet table is a two-column PCIe/SXM2 layout, so the SXM2 power reads as the second value (300 W); it shows only 'Mar18' as its date. The earliest window found is for Volta-based DGX systems, not the bare GPU.

1 dated promise and the shipment

each date links to the page it is quoted from
  1. 10 May 2017 Promised Volta-based DGX-1 and DGX Station orderable now, expected to ship in the third quarter in the third quarter of 2017. Volta-based DGX Station and DGX-1 servers are available now and expected to ship in the third quarter. NVIDIA Advances AI Computing Revolution with New Volta-Based DGX Systems
  2. 10 Aug 2017 Shipped: began shipping Tesla V100; shipped Volta in volume to AI customers in the quarter. This quarter, we shipped Volta in volume to leading AI customers. NVIDIA Announces Financial Results for Second Quarter Fiscal 2018

Named at no AI data center Epoch AI tracks

H100-equivalents and power are the whole site’s today, with every chip it holds

124,932 chips in 46 GPU clusters

Epoch AI’s list of AI supercomputers, which stops in July 2025; the 20 largest shown, all in the CSV of Epoch’s list
ClusterOwnerCountryChipsFirst operational
Meta 2017 V100 Cluster Meta AI United States of America 22,000 2017-10-01
Lawrence Livermore NL Sierra US Department of Energy United States of America 17,280 2018-06-01
Azure OpenAI GPT-3 Cluster Microsoft,OpenAI United States of America 10,000 2020-04-15
XTX Markets Cluster XTX Markets Not given 10,000 2023-03-27
Saudi Aramco Dammam-7 Saudi Aramco Saudi Arabia 7,912 2021-01-21
Eni HPC5 Eni Italy 7,280 2020-02-21
NVIDIA SATURN V Phase 2 Nvidia United States of America 5,280 2018-03-26
Cineca Marconi-100 Cineca Italy 3,952 2020-04-20
TotalEnergies Pangea III TotalEnergies France 3,384 2019-06-18
Lawrence Livermore NL Lassen Phase 2 US Department of Energy United States of America 3,168 2020-08-19
Eni HPC4 Phase 2 Eni Italy 2,750 2021-10-30
Jean Zay Supercomputer Phase 3 GENCI France 2,696 2022-06-01
Petrobras Dragão Petrobras Brazil 2,176 2021-06-01
AWS Fast BERT Training Amazon United States of America 2,048 2019-12-03
Taiwania 2 National Applied Research Laboratories Taiwan Taiwan 2,016 2018-11-01
NVIDIA DGX SuperPOD 2019 Nvidia Not given 1,536 2019-06-17
RPI AiMOS Rensselaer Polytechnic Institute United States of America 1,512 2019-12-11
G42 Artemis G42 United Arab Emirates 1,296 2019-11-01
Microsoft Azure Immunity Bio Microsoft United States of America 1,250 2020-04-01
SberCloud Christofari SberCloud Russia 1,200 2019-11-09

Epoch AI’s names for this chip: NVIDIA V100; NVIDIA Tesla V100 SXM2; NVIDIA Tesla V100 DGXS 32 GB.

Other NVIDIA chips

  • A100 SXM 80GB 780 gigaflops of dense BF16 compute per watt, at 400 W
  • H100 SXM 1,414 gigaflops of dense BF16 compute per watt, at 700 W
  • H200 SXM 1,414 gigaflops of dense BF16 compute per watt, at 700 W
  • GH200 Grace Hopper Superchip 990 gigaflops of dense BF16 compute per watt, at 1,000 W
  • B200 2,200 gigaflops of dense BF16 compute per watt, at 1,000 W
  • GB200 Grace Blackwell Superchip 5,000 teraflops of dense BF16 compute a chip; NVIDIA publishes no power figure
  • B300 (Blackwell Ultra) 2,000 gigaflops of dense BF16 compute per watt, at 1,100 W
  • Rubin GPU 4,000 teraflops of dense BF16 compute a chip; NVIDIA publishes no power figure

Sources

accelerators.csv Every dataset → Status → Glossary →