Advanced Computing in the Age of AI | Thursday, March 28, 2024

Google Gets Big Backers For Kubernetes Docker Manager 

The Kubernetes software container management system that search engine giant Google just open sourced a month ago has got some big – and perhaps surprising – backers. And that makes it more likely that Kubernetes could become a de facto standard and commercialized someday.

Kubernetes, an ancient Greek word meaning pilot, is not an open source version of the current Omega cluster management system that Google users internally to place jobs on its fleet of over 1 million servers, and it is not the code from Omega's predecessor, Borg, that has been let out into the wild. Rather, Kubernetes is a more modest tool for managing software containers on systems, and one that the company created based on ideas from Borg and Omega to set up containers on Google Compute Engine, the infrastructure portion of the search engine giant's public cloud.

Google runs on open source software as well as on custom system and application code that it creates, and it doesn't often open source whole programs, but made an exception with Kubernetes. As far as we know, everything at Google runs atop a Linux operating system and in control groups, or cgroups for short, which is a software container technology in Linux that allows for multiple workloads to run on a single physical system but without having to go all the way with a server virtualization hypervisor that has a lot more overhead. If everything in your stack runs your own version of Linux, containers are all you need.

Google is not about to open source either Borg or Omega, any more than it has open sourced its MapReduce, BigTable, or other data analytics technologies that underpin its business. But it has made an exception with the Kubernetes tool, and the reason why is simple: Google operates a public cloud now and it wants companies on Google Compute Platform to use a tool it understands and then, when they deploy a similar container controller internally, to use the same one.

Having Microsoft, IBM, Red Hat, and Docker offer to contribute to the Kubernetes project is a big deal, and it is interesting that Mesosphere, which has created a cluster management tool called Mesos based on the ideas behind Borg and which is used by Twitter, Airbnb, HubSpot, and others, has also joined the effort. So have CoreOS and SaltStack. The former company has created a minimalist Linux variant aimed at hyperscale computing, with clustering and containers built in, while the latter has built a configuration management tool for cloudy infrastructure that is akin to Puppet and Chef. Mesos and SaltStack are not precisely direct competitors to Kubernetes, but with Mesos there seems to be some overlap. Kubernetes has the ability to deploy, monitor, and replicate containers – specifically Docker containers, which like Linux containers in the current commercial releases (abbreviated LXC), are based on cgroups and other technologies that Google helped develop and harden.

Google has high hopes for Kubernetes and is trying to build a community around it, starting with the players outlined above. "Each company brings unique strengths," the company said in a blog announcement, "and together we will ensure that Kubernetes is a strong and open container management framework for any application and in any environment – whether in a private, public or hybrid cloud.

Google created Kubernetes to manage Docker containers on Google Compute Engine and the Managed VMs that are part of App Engine, and Microsoft is joining the Kubernetes community to make sure that it can manage Linux environments and their containers in the same way. Red Hat said in the blog announcement that it was "working to bring Kubernetes to the open hybrid cloud," and that probably means that if Kubernetes takes off it will get woven into Red Hat Enterprise Linux, its Satellite management tool, and possibly even Red Hat's version of the OpenStack cloud controller. IBM is working on an open governance model for Kubernetes – meaning, hiring the lawyers – and is something Google has experience with in conjunction with Big Blue after the two worked to create the OpenPower Foundation last summer that is opening up IBM's Power chip technology. Docker Software is going to aligning its roadmaps and upstream development with the Kubernetes project, despite the fact that it has its own tool, called Libswarm, that does essentially the same job. Mesosphere will be wrapping its Mesos cluster management tool, which does a lot more than boss around containers, around Kubernetes and it could turn out that it absorbs this code into its own stack at some point, but Mesos is mostly written in C++ and Kubernetes is mostly written in Go, so that may be a no-go. Both are under an Apache license, so integration would be relatively easy in that regard.

Kubernetes is still a project very much in its infancy, and Google warns that while it is based on its years of experience in designing and building large-scale cluster managers, it is still under heavy development and tire kickers and contributors should expect bugs as well as design and API changes as it is brought from beta to a stable product over the next year.

EnterpriseAI