Live
Local Artifacts Support Lets Workers Test Repo Workflows Before DeployCo‑Packaged Optics Cut Power Use and Latency in Lambda’s 10k‑GPU AI ClusterScaling AI Agents on Kubernetes: New CNCF Projects and Operational ImplicationsGitHub Copilot model deprecation forces updates to AI‑assisted development pipelinesGitHub Copilot expands Enterprise Server access and adds new AI models – practical impact for engineersArchitecting Agentic Customer Engagement with Amazon Bedrock AgentCoreBuilding Multi‑Tenant Memory‑Aware Agents with Bedrock Agentic Retrieve and Managed Knowledge BaseBulk‑Start AI Scan for Up to 20 GitHub Repos: What Engineers Need to KnowLocal Artifacts Support Lets Workers Test Repo Workflows Before DeployCo‑Packaged Optics Cut Power Use and Latency in Lambda’s 10k‑GPU AI ClusterScaling AI Agents on Kubernetes: New CNCF Projects and Operational ImplicationsGitHub Copilot model deprecation forces updates to AI‑assisted development pipelinesGitHub Copilot expands Enterprise Server access and adds new AI models – practical impact for engineersArchitecting Agentic Customer Engagement with Amazon Bedrock AgentCoreBuilding Multi‑Tenant Memory‑Aware Agents with Bedrock Agentic Retrieve and Managed Knowledge BaseBulk‑Start AI Scan for Up to 20 GitHub Repos: What Engineers Need to Know
NVIDIA

Co‑Packaged Optics Cut Power Use and Latency in Lambda’s 10k‑GPU AI Cluster

AI SummaryPowered by AI

Lambda swapped traditional pluggable transceivers for NVIDIA Photonics co‑packaged optics switches across its 10,000‑GPU training fabric. The change reduces latency, improves power efficiency, and raises link reliability, directly affecting AI, cloud, and SRE workloads.

Lambda has replaced conventional pluggable transceivers with NVIDIA Photonics co‑packaged optics (CPO) switches in the network that ties together more than 10,000 GPUs across GB300 NVL72 racks. The shift delivers measurable latency improvements, significant power savings, and higher component reliability—factors that matter to AI engineers, platform architects, and SRE teams responsible for large‑scale training clusters.

Performance and Power Impact

In Lambda’s tests, the CPO switches lowered network latency by roughly 4% at the rack level and up to 10% at pod scale, while preserving the same throughput as legacy pluggable optics. Power efficiency claims up to a five‑fold improvement, and the design uses four times fewer lasers, reducing overall power draw. Temperature measurements show a median optics temperature of 36 °C compared with 51 °C for traditional modules, indicating a cooler operating envelope that can translate into lower cooling overhead.

Reliability and Maintenance

The co‑packaged design is reported to be up to 15× more reliable than conventional optics. Fewer lasers and a tighter electrical‑to‑optical conversion path reduce failure points. Each switch houses 18 removable external light‑source modules that can be swapped in the field, avoiding full module replacement when a laser fails. The combination of lower temperature, fewer components, and higher mean time between interruptions (MTBI) directly mitigates the risk of a single link outage stalling thousands of GPUs.

Architectural Considerations

Co‑packaged optics integrate the optical engine directly on the switch package, shortening the electrical path from inches to millimeters. This eliminates the need for retimer chips, cutting both part count and power consumption. The switches provide 144 ports at 800 Gb/s each and are liquid‑cooled, meaning rack designs must accommodate coolant distribution and ensure adequate airflow for the switch chassis. Cabling strategies remain fiber‑based, but the reduced component count can simplify inventory and spares planning.

Operational Implications

From an SRE perspective, the new optics introduce new telemetry points: optics temperature, laser health, and module replacement status. Monitoring these metrics can provide early warning of degradation before a hard failure. Power budgeting should be revisited to account for the lower draw per port, potentially freeing capacity for additional compute. Incident response playbooks may need updates to include field replacement of the 18 light‑source modules without taking the entire switch offline.

Related CloudNinjas coverage: hands-on guides.

What This Means For Practitioners

  • When designing next‑generation AI training fabrics, evaluate CPO switches for power‑budget savings and latency gains.
  • Incorporate optics temperature and module health into existing monitoring dashboards.
  • Plan for field‑replaceable light‑source modules in maintenance procedures to reduce MTTR.
  • Reassess rack cooling designs to leverage the lower thermal envelope of co‑packaged optics.
  • Track reliability metrics (MTBI) to quantify the operational benefit of the 15× reliability claim.
Originally published atNVIDIA Blog