The NVIDIA Vera Rubin platform has set new performance records across the full MLPerf Training v6.1 suite and delivered a provisional ten‑fold improvement in throughput per watt for agentic inference workloads. For engineers responsible for building, scaling, and securing AI workloads, these numbers signal both opportunities for faster iteration and new operational constraints around power, networking, and software stack optimization.
Training Benchmarks and GPU‑Level Gains
Vera Rubin’s NVL72 system achieved results on all ten listed MLPerf Training v6.1 entries, covering large language models, image generation, recommender systems, and reinforcement learning. Compared with the prior‑generation NVIDIA GB300 NVL72, the new platform delivered up to 2× more performance per GPU. The benchmark suite includes entries such as 6.1‑0116 and 6.1‑0138, all reported on the MLCommons site on 16 Nov 2026.
Scaling Efficiency at Large GPU Counts
On the DeepSeek‑V3 benchmark, the NVL72 configuration scaled from 256 to 1,024 GPUs with roughly 97 % efficiency, cutting training time from 16 minutes to about four. NVIDIA also demonstrated runs on 10,240 GPUs using GB300 systems, a scale matched by Microsoft Azure with 10,240 GB200 GPUs. The efficiency at scale relies on NVIDIA NVLink for intra‑system GPU communication and Quantum‑X800 InfiniBand plus Spectrum‑X Ethernet for inter‑system data‑center networking. Practitioners must verify that their networking fabric can sustain similar efficiency when expanding clusters.
Agentic Inference Throughput per Watt
In the first MLPerf Endpoints submission, Vera Rubin NVL72 (32 accelerators) achieved a throughput of 41,414 tokens / sec at 100.89 kW, yielding a throughput‑per‑kilowatt advantage of more than ten times the GB300 NVL72 (48 accelerators, 2,210 tokens / sec at 60 kW). The comparison is based on a shared minimum interactivity of 50 tokens / sec per user, a metric that reflects real‑world agentic workloads such as code‑writing assistants or automated IT troubleshooting bots. These provisional results are pending MLCommons verification.
Architectural and Operational Implications
- Power budgeting: Higher throughput per watt can reduce the overall power envelope for a given workload, but the absolute power draw (≈100 kW for the NVL72 endpoint) remains substantial. Capacity planning must account for both peak and sustained power consumption.
- Network topology: Maintaining ~97 % scaling efficiency requires low‑latency, high‑bandwidth links (NVLink, Quantum‑X800 InfiniBand, Spectrum‑X Ethernet). Engineers should evaluate whether existing fabric can support similar scaling or if upgrades are needed.
- Software stack continuity: NVIDIA attributes continued performance gains to ongoing CUDA‑stack optimizations. Teams should track CUDA releases and integrate updated libraries to capture incremental improvements without full hardware refreshes.
- Workload placement: The dramatic throughput per watt advantage for agentic inference suggests consolidating many concurrent agents onto fewer, more efficient nodes, potentially simplifying autoscaling policies.
Related CloudNinjas coverage: AI engineering.
What This Means For Practitioners
Practitioners should benchmark their own training pipelines against the reported NVL72 figures to quantify real‑world gains, especially when migrating from older GB300‑based clusters. Verify that networking fabric can sustain near‑linear scaling before committing to multi‑thousand‑GPU deployments. For inference services, consider re‑architecting to exploit the higher token‑per‑watt density, which can lower operational costs and increase the number of concurrent agents per rack. Finally, keep the CUDA software stack up to date to benefit from the incremental performance improvements NVIDIA promises as the platform evolves.



