The networking industry is starting to embrace the Open Compute Project (OCP). While it’s probably too early to speculate how OCP networking will pan out, but it is hard to contain our excitement.
OCP is an industry association originating from the largest data center hardware customers. Together they represent a huge market and hardware manufacturers are falling in line to join and satisfy their needs. Luckily, the big guys’ needs align well with the rest of us. Everybody wants things to work simply, cheaply, and interchangeably.
“Hardware” means racks, mounts, and power distribution all the way through servers and storage to networking equipment. Participating companies share their in-house designs hoping they will catch on, become mass-manufactured COTS products, driving down prices with economies of scale.
On the networking front, aside from hardware aspects like form factor and power, interchangeability is taken to new levels: All networking equipment (switches, routers, firewalls, load balancers, etc.) is expected to be available for purchase as “Bare Metal” hardware, with an open bootloader, and the ability to install a third-party “Network Operating System”. The same way we can purchase generic x86 servers today and install any OS and applications of our choosing, in the future we might be able to purchase networking hardware from Juniper and run Cisco’s IOS on it. We are quite far from such utopia, and it may never come to be, but there is a bewildering variety of networking equipment available, a nice selection of NOS-es, including an open source Linux distribution to take as a base and roll your own.
The key to compatibility is ONIE, the Open Network Installation Environment bootloader.
The uptake from network equipment manufacturers (often abbreviated as ODMs, Original Device Manufacturers) is huge: Alongside their traditional, integrated products, they now offer many models of ONIE-capable switches, routers, appliances, network-server hybrids. Many more are being demonstrated as prototypes at trade shows such as the OCP Summit.
Despite the variety of appearances, port counts and speeds, throughput and capabilities, all these boxes share roughly the same internal design principles. There is the host processor part (often an Atom-class x86 board, but it can be PPC or ARM), and the switching silicon (a Broadcom Trident variant, a Cavium Xpliant, or similar). The ASIC (or more accurately, ASSP, Application Specific Standard Product) is usually connected to the PCIe bus of the host. The host often has more non-volatile storage than regular networking equipment — onboard Flash, SD card, or SSD. Neither has a significant price impact nowadays.
Hardware is sold either bundled with an NOS license which is then preinstalled, but is replaceable, or with only ONIE.
Aside from ONIE, the minimal Linux boot environment which enables the installation of a NOS, the switch vendors must provide the means to configure or program their networking silicon.
Traditionally, suppliers like Broadcom and Cavium have kept their SDKs and toolchains under tight control, available only to their partners including ODMs and select expert software development consulting service providers like Northforge Innovations. Being low-level APIs, having access to them would be of little value to organizations without teams of dedicated engineers with specialized experience with each vendor and product.
A welcome change to this state of affairs is the introduction of open APIs from the vendors: OF-DPA, OpenNSL, Switch Abstraction Interface, etc. These open SDKs are accessible for a much wider audience both from the availability and ease of use point of view: They are freely downloadable as a bundle of a closed-source binary and open-source headers, and the APIs presented are typically higher level. The trade-off here is simplified usability at the cost of less low-level, direct control. The exposed functionality while limited, matches fairly well what a custom application developer might want to do with a Whitebox switch: OF-DPA for example provides an API closely resembling the OpenFlow table hierarchy, making it an obvious choice for any OpenFlow agent type application. The open SDKs are evolving, the vendors are listening, and their classic SDK partners have the means to extend and enhance the offerings.
Network Operating Systems are built on top of the foundations of ONIE and the open or closed SDKs. Typically, these NOSes are custom Linux distributions containing:
- An ONIE-compatible installer,
- Driver support for the special hardware in the Whitebox switch including GPIO and I2C access to LEDs, SFPs, power- and thermal management,
- And most importantly, the PCIe interface of the switching ASIC.
The free Open Network Linux distribution, a development platform for custom applications, stops short right there. Full-feature NOS-es typically also include applications to do routing, switching, monitoring, and/or firewalling. Complete with OAM features, when (pre-) installed on a Whitebox switch they offer a polished, ready-made customer experience comparable to that of buying classical, non-Whitebox networking equipment from the likes of Cisco or Juniper. Examples include Cumulus Linux, BigSwitch SwitchLight OS, Pica8 PicOS, or even the free and open source OpenSwitch NOS founded by HP and other leading networking industry companies.
The alternative to these turnkey NOS-es is rolling your own.
Take, for example, Open Network Linux as the base, pick the SDK for your target platform and application, and implement your own routing/switching logic. Open Source components can be cherry-picked for vanilla inclusion or forking to add the custom functionality. Hyperscalers (the Googles and Facebooks of the world) have the talent and resources to do it in house, smaller players can take advantage of the growing ecosystem of consulting or software development services.
It is probably too early to speculate how OCP networking will pan out, but it is hard to contain our excitement.
Will it remain the purview of only the biggest players? Will a handful of NOS offerings cover every conceivable need and obviate independent development? Or is an “ONIE compatible switch” our generation’s “IBM compatible PC”?
OCP Summit 2016
The 2016 OCP Summit in San Jose was altogether a relatively big event for such a fresh and specialized industry group as the Open Compute Project. The tradeshow floor, talks, workshops, and announcements were all split across the many areas of interest involved, of which Networking is only one. Still, the two days commanded a rather tight schedule to meet all the network-related vendors, see the most relevant talks, and drop by the Hackaton.
Many vendors and member companies were represented at the CEO, CTO or VP level, indicating their interest in this promising new field, and acknowledging the Summit was more than just a trade show. We all went there not just to promote ourselves, but also to be part of the discussion on where OCP is headed, what new applications it will enable, and how we can all work together to make it pan out and succeed.
There was an almost tangible, but hard to describe anticipation in the air. We all felt the Summit is not so much about one-upping each other, but more about being part of something big and genuinely useful for everyone whose life is impacted by technology.
Northforge Innovations developed networking applications with switch ODMs and switch ASIC providers long before the OCP movement. Our long-term relationships and experience with Broadcom, Cavium, Mellanox and Intel enables us to provide solutions and services on Whitebox platforms just like we’ve been doing on closed platforms. We go beyond OF-DPA, OpenNSL and OpenXPS, typically working with the underlying SDKs accessible only to select partner companies.
The Open Compute Project promises to open up this fascinating field to newcomers. We applaud and support these efforts, wish to welcome everyone to our world, and offer our services and partnership to anyone in need of in-depth expertise in this domain.
Author: Gabor Sz