NVIDIA H100 SXM · Hopper

BF16, dense

990 TFLOPS BFLOAT16 Tensor Core* 1,979 teraFLOPS NVIDIA H100 Tensor Core GPU (halved, because the page gives it with sparsity)

FP8, dense

1,979 TFLOPS FP8 Tensor Core* 3,958 teraFLOPS NVIDIA H100 Tensor Core GPU (halved, because the page gives it with sparsity)

Memory

80 GB GPU Memory 80GB 94GB NVIDIA H100 Tensor Core GPU (read from a table with a column for each product, whose head reads “Product Specifications H100 SXM H100 NVL”)

Memory bandwidth

3.35 TB/s GPU Memory Bandwidth 3.35TB/s NVIDIA H100 Tensor Core GPU

Maximum power

700 W Max Thermal Design Power (TDP) Up to 700W (configurable) NVIDIA H100 Tensor Core GPU

Compute per watt

1,414 GFLOPS/Wdense BF16 over maximum power; 2,827 GFLOPS/W at FP8

The H100 page prints only sparse Tensor Core figures (marked *); dense is half.

1. 22 Mar 2022 Promised H100 available starting in the third quarter in the third quarter of 2022. NVIDIA H100 will be available starting in the third quarter. NVIDIA Announces Hopper Architecture, the Next Generation of Accelerated Computing
2. 20 Sep 2022 Shipped: H100 in full production; partner products from October. NVIDIA today announced that the NVIDIA H100 Tensor Core GPU is in full production NVIDIA Hopper in Full Production

Epoch AI’s names for this chip: H100; NVIDIA H100 SXM5 80GB.

• Tesla V100 SXM2 417 gigaflops of dense FP16 compute per watt, at 300 W
• A100 SXM 80GB 780 gigaflops of dense BF16 compute per watt, at 400 W
• H200 SXM 1,414 gigaflops of dense BF16 compute per watt, at 700 W
• GH200 Grace Hopper Superchip 990 gigaflops of dense BF16 compute per watt, at 1,000 W
• B200 2,200 gigaflops of dense BF16 compute per watt, at 1,000 W
• GB200 Grace Blackwell Superchip 5,000 teraflops of dense BF16 compute a chip; NVIDIA publishes no power figure
• B300 (Blackwell Ultra) 2,000 gigaflops of dense BF16 compute per watt, at 1,100 W
• Rubin GPU 4,000 teraflops of dense BF16 compute a chip; NVIDIA publishes no power figure