Chips · accelerators · Google

Google TPU v4 · TPU

275 teraflops of dense BF16 compute a chip; Google publishes no power figure

Shipped 12 Oct 2022; shipped after the first date promised, up to 10 months. Epoch AI names it at 2 AI data centers and 4 GPU clusters. Google across the site.

Embed

The frame credits Google, Epoch AI, AI data centers and Epoch AI, Data on GPU clusters and links back to its page. Preview it.

Cite
CSV Post on X

Each figure in Google’s own words

one chip, dense compute; memory is high-bandwidth memory
BF16, dense
275 TFLOPS Peak compute per chip 275 teraflops (bf16 or int8) TPU v4
INT8, dense
275 TOPS Peak compute per chip 275 teraflops (bf16 or int8) TPU v4
Memory
32 GiB HBM2 capacity and bandwidth 32 GiB, 1200 GBps TPU v4
Memory bandwidth
1.2 TB/s HBM2 capacity and bandwidth 32 GiB, 1200 GBps TPU v4 (1200 GB/s on the page)
List price
$3.22 a chip-hourTPU v4 pod us-central2 Oklahoma $3.22 Cloud TPU pricing (on-demand, us-central2 (Oklahoma), TPU v4 pod, per chip-hour); 85 TFLOP-hours of dense 16-bit compute a dollar

Google does not publish a TDP for TPU v4; the v4 page gives only measured power (90/170/192 W min/mean/max), so power is left out. TPU v4 Pods went to preview on 2022-05-12 and to general availability on 2022-10-12, after the 2021 window.

1 dated promise and the shipment

each date links to the page it is quoted from
  1. 18 May 2021 Promised TPU v4 pods available to Cloud customers in 2021. They'll be available to our Cloud customers later this year. Google I/O 2021: Being helpful in moments that matter
  2. 12 Oct 2022 Shipped: Cloud TPU v4 Pods generally available. Cloud TPU v4 Pods - Now in General Availability (GA) Google Cloud infrastructure enhancements tailored for your workloads

Named at 2 AI data centers

H100-equivalents and power are the whole site’s today, with every chip it holds
AI data centers that name this chip
SiteOwnerCountryH100eMWChips
Google Storey County Google United States 133,906 214 TPU v5e, TPU v4, TPU v6e, TPU v5p
Google Papillion Google United States 100,556 284 TPU v4, TPU v5e, TPU v5p

71,680 chips in 4 GPU clusters

Epoch AI’s list of AI supercomputers, which stops in July 2025
GPU clusters with this chip
ClusterOwnerCountryChipsFirst operational
Google Oklahoma TPU v4 Pods Google United States of America 32,768 2022-05-12
Gemini 1.0 Ultra training cluster A Google United States of America 28,672 2023-08-20
Paper on PaLM Google United States of America 6,144 2022-04-05
Google TPU v4 Pod Google United States of America 4,096 2021-05-19

Epoch AI’s names for this chip: TPU v4; Google TPU v4.

Other Google chips

  • TPU v5e 197 teraflops of dense BF16 compute a chip; Google publishes no power figure
  • TPU v5p 459 teraflops of dense BF16 compute a chip; Google publishes no power figure
  • TPU v6e 918 teraflops of dense BF16 compute a chip; Google publishes no power figure
  • TPU7x (Ironwood) 2,307 teraflops of dense BF16 compute a chip; Google publishes no power figure