A long-awaited, emerging computer network component may finally be having its moment. At Nvidia’s GTC event last week in San Jose, the company announced that it will produce an optical network switch designed to drastically cut the power consumption of AI data centers. The system—called a co-packaged optics, or CPO, switch—can route tens of terabits per second from computers in one rack to computers in another. At the same time, startup Micas Networks, announced that it is in volume production with a CPO switch based on Broadcom’s technology.

In data centers today, network switches in a rack of computers consist of specialized chips electrically linked to optical transceivers that plug into the system. (Connections within a rack are electrical, but several startups hope to change this.) The pluggable transceivers combine lasers, optical circuits, digital signal processors, and other electronics. They make an electrical link to the switch and translate data between electronic bits on the switch side and photons that fly through the data center along optical fibers.

Co-packaged optics is an effort to boost bandwidth and reduce power consumption by moving the optical/electrical data conversion as close as possible to the switch chip. This simplifies the setup and saves power by reducing the number of separate components needed and the distance electronic signals must travel. Advanced packaging technology allows chipmakers to surround the network chip with several silicon optical transceiver chiplets. Optical fibers attach directly to the package. So, all the components are integrated into a single package except for the lasers, which remain external because they are made using non-silicon materials and technologies. (Even so, CPOs require only one laser for every eight data links in Nvidia’s hardware.)

“An AI supercomputer with 400,000 GPUs is actually a 24-megawatt laser.” —Ian Buck, Nvidia

As attractive of a technology as that seems, its economics have kept it from deployment. “We’ve been waiting for CPO forever,” says Clint Schow, a co-packaged optics expert and IEEE Fellow at the University of California Santa Barbara, who has been researching the technology for 20 years. Speaking of Nvidia’s endorsement of technology, he said the company “wouldn’t do it unless the time was here when [GPU-heavy data centers] can’t afford to spend the power.” The engineering involved is so complex, Schow doesn’t think it’s worthwhile unless “doing things the old way is broken.”

And indeed, Nvidia pointed to power consumption in upcoming AI data centers as a motivation. Pluggable optics consume “a staggering 10 percent of the total GPU compute power” in an AI data center, says Ian Buck, Nvidia’s vice president of hyperscale and high-performance computing. In a 400,000-GPU factory that would translate to 40 megawatts, and more than half of that goes just to powering the lasers in a pluggable optics transceiver. “An AI supercomputer with 400,000 GPUs is actually a 24-megawatt laser,” he says.

Optical Modulators

One fundamental difference between Broadcom’s scheme and Nvidia’s is the optical modulator technology that encodes electronic bits onto beams of light. In silicon photonics there are two main types of modulators—Mach-Zender, which Broadcom uses and is the basis for pluggable optics, and microring resonator, which Nvidia chose. In the former, light traveling through a waveguide is split into two parallel arms. Each arm can then be modulated by an applied electric field, which changes the phase of the light passing through. The arms then rejoin to form a single waveguide. Depending on whether the two signals are now in phase or out of phase, they will cancel each other out or combine. And so electronic bits can be encoded onto the light.

Microring modulators are far more compact. Instead of splitting the light along two parallel paths, a ring-shaped waveguide hangs off the side of the light’s main path. If the light is of a wavelength that can form a standing wave in the ring, it will be siphoned off, filtering that wavelength out of the main waveguide. Exactly which wavelength resonates with the ring depends on the structure’s refractive index, which can be electronically manipulated.

However, the microring’s compactness comes with a cost. Microring modulators are sensitive to temperature, so each one requires a built-in heating circuit, which must be carefully controlled and consumes power. On the other hand, Mach-Zender devices are considerably larger, leading to more lost light and some design issues, says Schow.

That Nvidia managed to commercialize a microring-based silicon photonics engine is “an amazing engineering feat,” says Schow.

Nvidia CPO Switches

According to Nvidia, adopting the CPO switches in a new AI data center would lead to one-fourth the number of lasers, boost power efficiency for trafficking data 3.5-fold, improve the reliability of signals making it from one computer to another on time by 63-times, make networks 10-fold more resilient to disruptions, and allow customers to deploy new data center hardware 30 percent faster.

“By integrating silicon photonics directly into switches, Nvidia is shattering the old limitation of hyperscale and enterprise networks and opening the gate to million-GPU AI factories,” said Nvidia CEO Jensen Huang.

– YouTubeyoutu.be

The company plans two classes of switch, Spectrum-X and Quantum-X. Quantum-X, which the company says will be available later this year, is based on Infiniband network technology, a network scheme more oriented to high-performance computing. It delivers 800 Gb/s from each of 144 ports, and its two CPO chips are liquid-cooled instead of air-cooled, as are an increasing fraction of new AI data centers. The network ASIC includes Nvidia’s SHARP FP8 technology, which allows CPUs and GPUs to offload certain tasks to the network chip.

Spectrum-X is an Ethernet-based switch that can deliver a total bandwidth of about 100 terabits per second from a total of either 128 or 512 ports and 400 Tb/s from 512 or 2048 ports. Hardware makers are expected to have Spectrum-X switches ready in 2026.

Nvidia has been working on the fundamental photonics technology for years. But it took collaboration with 11 partners—including TSMC, Corning, and Foxconn—to get the switch to a commercial state.

Ashkan Seyedi, director of optical interconnect products at Nvidia, stressed how important it was that the technologies these partners brought to the table were co-optimized to satisfy AI data center needs rather than simply assembled from those partners’ existing technologies.

“The innovations and the power savings enabled by CPO are intimately tied to your packaging scheme, your packaging partners, your packaging flow,” Seyedi says. “The novelty is not just in the optical components directly, it’s in how they are packaged in a high-yield, testable way that you can manage at good cost.”

Testing is particularly important, because the system is an integration of so many expensive components. For example, there are 18 silicon photonics chiplets in each of the two CPOs in the Quantum-X system. And each of those must connect to two lasers and 16 optical fibers. Seyedi says the team had to develop several new test procedures to get it right and trace where errors were creeping in.

Micas Networks Switches

Micas Networks is already in production with a switch based on Broadcom’s CPO technology.Micas Network

Broadcom chose the more established Mach-Zender modulators for its Bailly CPO switch, in part because it is a more standardized technology, potentially making it easier to integrate with existing pluggable transceiver infrastructure, explains Robert Hannah, senior manager of product marketing in Broadcom’s optical systems division.

Micas’ system uses a single CPO component, which is made up of Broadcom’s Tomahawk 5 Ethernet switch chip surrounded by eight 6.4 Tb/s silicon photonics optical engines. The air-cooled hardware is in full production now, putting it ahead of Nvidia’s CPO switches.

Hannah calls Nvidia’s involvement an endorsement of Micas’ and Broadcom’s timing. “Several years ago, we made the decision to skate to where the puck was going to be,” says Mitch Galbraith, Micas’ chief operations officer. With data center operators scrambling to power their infrastructure, CPO’s time seems to have come, he says.

The new switch promises a 40 percent power savings versus systems populated with standard pluggable transceivers. However, Charlie Hou, vice president of corporate strategy at Micas, says CPO’s higher reliability is just as important. “Link flap,” the term for transient failure of pluggable optical links, is one of the culprits responsible for lengthening already-very-long AI training runs, he says. CPO is expected to have less link flap because there are fewer components in the signal’s path, among other reasons.

CPOs in the Future

The big power saving data centers are looking to get from CPO is mostly a one-time benefit, Schow suggests. After that, “I think it’s just going to be the new normal.” However, improvements to the electronics’ other features will let CPO makers keep boosting bandwidth—for a time at least.

Schow doubts individual silicon modulators—which run at 200 Gb/s in Nvidia’s photonic engines—will be able to go past much more than 400 Gb/s. However, other materials, such as lithium niobate and indium phosphide should be able to exceed that. The trick will be affordably integrating them with silicon components, something Santa Barbara-based OpenLight is working on, among other groups.