At GTC 2026 in San Jose (March 16), Nvidia CEO Jensen Huang put the Vera Rubin platform at the centre of his keynote. Rubin is the successor to Blackwell (B200/B300), which itself replaced the Hopper generation (H100/H200). Rather than reducing the industry’s dependence on high-bandwidth memory, Rubin doubles down on it: each Rubin GPU carries 288 GB of next-generation HBM4. What Nvidia promises to cut is not memory, but the cost per generated token.

The Rubin Architecture

According to Nvidia’s published specifications, the Rubin GPU is built on TSMC’s 3 nm-class process as a multi-chip design: two near-reticle-size compute dies plus I/O dies on a CoWoS-L interposer, with eight HBM4 stacks — around 336 billion transistors in total. Headline figures are 288 GB of HBM4 with roughly 22 TB/s of memory bandwidth and about 50 petaFLOPS of NVFP4 (4-bit floating point) inference compute per GPU, several times the FP4 throughput of a B200.

Rubin is paired with Nvidia’s new Vera CPU, an 88-core Arm-based processor that replaces Grace. GPUs and CPUs are connected through NVLink 6, and networking comes from ConnectX-9 SuperNICs, BlueField-4 DPUs and Spectrum-X switches — including co-packaged optics versions. Nvidia said the platform’s seven chips were all in full production.

Memory: More HBM, Not Less

Some early coverage suggested Rubin would lower costs by moving work from HBM to ordinary DDR5. That is not what Nvidia announced. The GPU’s working memory is HBM4 only; the Vera CPU uses its own LPDDR5X memory, as Grace did before it, and the CPU and GPU can share data over the coherent NVLink-C2C link. The demand for HBM — and the tight supply from SK hynix, Samsung and Micron — therefore grows with Rubin rather than shrinking.

Rack-Scale Performance

Nvidia sells Rubin primarily as complete rack-scale systems. The flagship Vera Rubin NVL72 combines 72 Rubin GPUs and 36 Vera CPUs in one liquid-cooled rack and is rated at about 3.6 exaFLOPS of NVFP4 inference. Nvidia’s headline claim, first made at CES in January 2026, is an up to 10× lower cost per inference token compared with Blackwell at system scale. These are vendor figures; independent benchmarks (for example MLPerf results) will only appear once systems ship.

GTC 2026 also brought the Groq 3 LPX rack — SRAM-based inference accelerators from the Groq technology Nvidia licensed in December 2025 — which Nvidia positions alongside Rubin for latency-critical, low-cost token generation.

Economic Impact on Data Centers

For cloud providers the attraction is throughput per watt and per dollar: if the token-cost claim holds, the same power budget serves far more AI requests. The flip side is that Rubin racks draw even more power and require liquid cooling, so operators must upgrade facilities rather than drop new chips into old racks. Rubin systems are scheduled to reach customers from the second half of 2026, with the larger Rubin Ultra generation planned for 2027.

Software and Ecosystem

As with previous generations, Rubin is programmed through CUDA and Nvidia’s software stack (TensorRT-LLM, Dynamo, NIM microservices), so code written for Hopper or Blackwell carries over. The main software work for customers is adopting low-precision formats such as NVFP4, which deliver most of Rubin’s quoted speed-ups.

Conclusion

Rubin is not a cheaper, HBM-light alternative to the H200 — it is the high end of Nvidia’s roadmap, succeeding Blackwell with more and faster HBM4, a new Arm CPU and rack-scale integration. Whether it really cuts token costs tenfold will be decided by real deployments later in 2026.

Sources: CNBC, GTC 2026 keynote; Guru3D, Rubin architecture details; StorageReview, GTC 2026