Advanced Computing in the Age of AI | Thursday, October 6, 2022

Prototype Open Compute Switches Revealed 

The Open Compute Project started by Facebook to do open source hardware may have to change its name soon, now that its Networking project to disaggregate switches and create an open ecosystem of hardware and software components is picking up steam.

The Networking project was established six months ago, and unlike other projects to date, this particular effort was not started by Facebook donating the design of piece of hardware, such as a datacenter, a server, a rack, or a storage array. Rather, the OCP sought input from large customers like financial services giants Goldman Sachs and Fidelity Investments as well as large service providers and is letting the community figure out what components to start with to create an open switch and an open stack of software to run atop of it. And the many of the big players in the switch market are helping, too. This effort is something that Najam Ahmad, director of technical operations at Facebook, explained to EnterpriseTech at length six weeks ago during Interop.

The idea is, simply put, is to do to switches what has been done to servers and to a lesser extent to storage arrays. To open them completely up and commoditize the parts to make switching less expensive, but more importantly, to make switches something that companies like Facebook can hack for their own purposes.

Frank Frankovsky, chairman of the OCP as well as vice president of hardware design and supply chain at Facebook, hosting an event in the company's Menlo Park headquarters to reveal some of the contributions that the vendor community is making to foster the creation of a whitebox switch and an open source stack, which anyone can learn about and which has no secret APIs, to run atop it.

"Six months ago, we realized when we looked up that we had these lovingly crafted islands of open source technology in all of our datacenters, but the things that were connecting them all were these proprietary black boxes," explained Frankovksy. "So we figured, what the heck, nothing in the datacenter stack should be immune from the positive impact of open source. When I say that people are inherently curious, I think that transparency enables that curiosity. The people that are around the Open Compute Project are the ones that are thinking differently about contributing technology that used to be considered proprietary. But then there is a whole other part of the ecosystem that wants to bring these pieces together to create solutions for customers. We are seeing a ton of momentum happening around this, and I think it is safe to say that we didn't know where this open networking project was going to go because we didn't have any preconceptions."

The switch vendor community is lining up quickly to participate, with the exception of Cisco Systems and the incumbent server makers who also have switch businesses to protect, such as Dell, Hewlett-Packard, and IBM. But the three major makers of merchant switch chips – Broadcom, Mellanox, and Intel – have all put together reference specifications for open switches and have submitting them to OCP as well as prototype hardware.

"I don't see anybody on the sidelines waiting for when this is going to start," said Frankovsky. "When we first started with servers, vendors said they guess they had better be in. With storage, we saw vendors think a little more quickly. With networking, it is clear that the time to execute is shrinking."

Broadcom is submitting an open switch based on its "Trident-II" ASIC, and it was the first switch chip maker to do so. The Broadcom reference architecture will sport 48 10Gb/sec ports and anywhere from four to twelve 40 Gb/sec ports. The switch will include a hardened Linux operating system and a software development kit supplied by Broadcom.

Mellanox has a specification based on its SwitchX-2 network chip, and the top of rack specification has 48 SFP+ ports and 12 QSFP ports running at 10 Gb/sec speeds. The design allows for the use of breakout cables for applications with less bandwidth demands.

The Mellanox reference switch will use the Open Network Install Environment, contributed by Cumulus Networks, which is an installer to change the network operating system on a switch. One of the dreams that Facebook has, and that the OCP members are embracing, is the ability to change network operating systems on a machine at will, much as can be done with servers today. It stands to reason that all OCP switches will support this ONIE tool eventually.

Intel's SeaCliff Trail OCP reference switch

Intel's SeaCliff Trail OCP reference switch


Intel is putting forth its SeaCliff Trail reference switch, which it revealed a little more than a year ago, as its OCP reference switch. This is based on the Fulcrum FM6764 ASIC from Intel, which supports VXLAN and NVGRE Layer 3 overlays to allow for Layer 2 networks to expand beyond the 4,096 VLAN limit. This reference switch has a Core processor that is used to do deep packet inspection, packet acceleration, encryption/decryption, and other work inside the switch. This runs a hardened version of Wind River Linux in Intel's reference architecture, but it is unclear what will be on the OCP design. This switch has 48 ports that run at 10 Gb/sec and four ports that run at 40 Gb/sec. Intel has already lined up original design manufacturers Quanta and Accton to build these devices on behalf of OCP members and anyone else who wants them, for that matter.

All three vendors have given machines to Facebook to be tested in its labs, and while Ahmad made no promises about when they would be deployed inside of Facebook's datacenters. When asked about when the company would be done with commercial switches he did say "I would like for Facebook to get 100 percent there."

When that might happen, Ahmad would not speculate. But it will probably not take years.

Add a Comment