Chips · accelerators · NVIDIA
NVIDIA A100 SXM 80GB · Ampere
780 gigaflops of dense BF16 compute per watt, at 400 W
Promised for the first half of 2021; past its date, no dated shipment on record. Epoch AI names it at 3 AI data centers and 89 GPU clusters.
Embed
The frame credits NVIDIA, Epoch AI, AI data centers and Epoch AI, Data on GPU clusters and links back to its page. Preview it.
Cite
Each figure in NVIDIA’s own words
one chip, dense compute; memory is high-bandwidth memory- BF16, dense
- 312 TFLOPS
BFLOAT16 Tensor Core 312 TFLOPS | 624 TFLOPS*
NVIDIA A100 Tensor Core GPU - INT8, dense
- 624 TOPS
INT8 Tensor Core 624 TOPS | 1248 TOPS*
NVIDIA A100 Tensor Core GPU - Memory
- 80 GB
GPU Memory 80GB HBM2e 80GB HBM2e
NVIDIA A100 Tensor Core GPU (read from a table with a column for each product, whose head reads “A100 80GB PCIe A100 80GB SXM”) - Memory bandwidth
- 2.04 TB/s
GPU Memory Bandwidth 1,935 GB/s 2,039 GB/s
NVIDIA A100 Tensor Core GPU (2,039 GB/s on the page; read from a table with a column for each product, whose head reads “A100 80GB PCIe A100 80GB SXM”) - Maximum power
- 400 W
Max Thermal Design Power (TDP) 300W 400W
NVIDIA A100 Tensor Core GPU (read from a table with a column for each product, whose head reads “A100 80GB PCIe A100 80GB SXM”) - Compute per watt
- 780 GFLOPS/Wdense BF16 over maximum power; 1,560 GFLOPS/W at INT8
A100 has no FP8, so int8 is given. The page gives dense and sparse (*); dense figures are used. No NVIDIA page found says the 80GB part itself began shipping: the 2020-05-14 'in full production and shipping' release is for the first (40GB) A100, so delivered is null.
2 dated promises
each date links to the page it is quoted from- 16 Nov 2020 Promised A100 80GB in DGX A100 and DGX Station A100, expected to ship this quarter in the fourth quarter of 2020.
The NVIDIA A100 80GB GPU is available in NVIDIA DGXTM A100 and NVIDIA DGX StationTM A100 systems, also announced today and expected to ship this quarter.
NVIDIA Doubles Down: Announces A100 80GB GPU - 16 Nov 2020 Promised partner HGX A100 systems with A100 80GB in the first half of 2021.
The NVIDIA A100 80GB GPU is available in NVIDIA DGXTM A100 and NVIDIA DGX StationTM A100 systems, also announced today and expected to ship this quarter.
NVIDIA Doubles Down: Announces A100 80GB GPU
Named at 3 AI data centers
H100-equivalents and power are the whole site’s today, with every chip it holds| Site | Owner | Country | H100e | MW | Chips |
|---|---|---|---|---|---|
| Bridge Data Centres MY06 | ByteDance | Malaysia | 221,052 | 315 | A100, H100, B200 |
| Microsoft Project Osmium | Microsoft | United States | 155,634 | 228 | A100, H100, B200 |
| xAI QTS Atlanta | SpaceXAI | United States | 11,470 | 24 | H100, A100 |
209,706 chips in 89 GPU clusters
Epoch AI’s list of AI supercomputers, which stops in July 2025; the 20 largest shown, all in the CSV of Epoch’s list| Cluster | Owner | Country | Chips | First operational |
|---|---|---|---|---|
| Oracle OCI Supercluster A100s | Oracle | United States of America | 32,768 | 2023-11-15 |
| Microsoft GPT-4 cluster | Microsoft,OpenAI | United States of America | 25,000 | 2022-04-30 |
| Meta Research SuperCluster (RSC-1) Phase 2 | Meta AI | United States of America | 16,000 | 2023-05-18 |
| EuroHPC Leonardo | EuroHPC JU | Italy | 13,824 | 2022-11-22 |
| Amazon Titan training cluster | Amazon | United States of America | 13,760 | 2023-01-15 |
| XTX Markets Cluster | XTX Markets | Not given | 10,000 | 2023-03-27 |
| Meta Research SuperCluster 2 (RSC-2) | Meta AI | United States of America | 8,000 | 2023-10-31 |
| Tesla A100 Cluster Phase 2 | Tesla | United States of America | 7,360 | 2022-08-16 |
| Lawrence Berkeley NL NERSC Perlmutter | US Department of Energy | United States of America | 6,144 | 2021-05-17 |
| Microsoft Azure Meta AI Rental | Microsoft | United States of America | 5,400 | 2022-05-27 |
| NVIDIA Selene Phase 2 | Nvidia | United States of America | 4,480 | 2020-11-30 |
| Tesla Training Cluster | Tesla | United States of America | 4,032 | 2021-08-20 |
| Ezra-1 Stability AI AWS Cluster | Amazon | Not given | 4,000 | 2022-07-20 |
| JUWELS-Booster | Julich Supercomputing Center | Germany | 3,744 | 2020-11-16 |
| Argonne NL Polaris | US Department of Energy | United States of America | 2,240 | 2021-11-01 |
| NAVER Corp Sejong | NAVER | South Korea | 2,240 | 2023-11-15 |
| Microsoft Azure Voyager-EUS2 | Microsoft | United States of America | 2,112 | 2021-11-01 |
| Softbank SuperPOD | Softbank | Japan | 2,048 | 2023-09-30 |
| Petrobras Pegasus (Pégaso) | Petrobras | Brazil | 2,016 | 2022-11-01 |
| ExxonMobil Discovery 5 | ExxonMobil | United States of America | 2,000 | 2022-11-01 |
Epoch AI’s names for this chip: A100; NVIDIA A100; NVIDIA A100 SXM4 80 GB; NVIDIA A100 SXM4 40 GB; NVIDIA A100 PCIe.
Other NVIDIA chips
- Tesla V100 SXM2 417 gigaflops of dense FP16 compute per watt, at 300 W
- H100 SXM 1,414 gigaflops of dense BF16 compute per watt, at 700 W
- H200 SXM 1,414 gigaflops of dense BF16 compute per watt, at 700 W
- GH200 Grace Hopper Superchip 990 gigaflops of dense BF16 compute per watt, at 1,000 W
- B200 2,200 gigaflops of dense BF16 compute per watt, at 1,000 W
- GB200 Grace Blackwell Superchip 5,000 teraflops of dense BF16 compute a chip; NVIDIA publishes no power figure
- B300 (Blackwell Ultra) 2,000 gigaflops of dense BF16 compute per watt, at 1,100 W
- Rubin GPU 4,000 teraflops of dense BF16 compute a chip; NVIDIA publishes no power figure
- NVIDIA’s own pages, each linked beside the figure or date it gives. Each page was read and each quote checked on 23 Oct 2026.
- Where it runs: Epoch AI, AI data centers and Epoch AI, GPU clusters, CC BY 4.0.