Advanced Computing in the Age of AI | Saturday, September 23, 2023

Docker Software Containers Go Commercial 

The next wave of virtualization, and one that has the potential to displace hypervisor-based virtualization on Linux platforms, is upon us now that Docker, the software container and application packaging system originally created by platform cloud provider dotCloud, has reached the 1.0 release milestone.

The Docker project is relatively new, being only fifteen months old, and a new company launched earlier this year is steering the development effort by 460 community contributors and, starting today, is providing the commercial-grade support that is always a prerequisite before any new technology – whether its source code is open or closed – can be adopted in the enterprise.

Docker, the company, is hosting its DockerCon user and techie conference in San Francisco this week and used the event to launch the Docker 1.0 software stack and its plans for providing enterprise support for the containerization technology.

The Docker stack includes Docker Engine, which provides a layer of abstraction above a server host operating system much as the kernel does on a bare metal hypervisor. But this is a much thinner layer of abstraction, which just wraps around the applications and their required binaries and libraries. This software container is similar in concept to Solaris zones in that Unix operating system or the new LXC containers that are in Linux. When Docker first got started, in fact, it rode atop LXC, but a few months ago the Docker community, working with Red Hat and Google, created a new container called libcontainer that reaches down into the cgroups and namespaces layers that underpin LXC directly, making for more streamlined operation. Customers can still use LXC containers, which are a plug-in option for Docker, and no doubt other types of containers may eventually be available for Linux. Docker also allows for different file systems to be plugged in underneath the containers, including BTRFS and AuFS as well as Device Mapper.

The Docker Engine will continue to be a free and open source tool, Scott Johnston, the new senior vice president of products who had a similar role at system configuration management upstart Puppet Labs, explains to EnterpriseTech. Docker, the company, will sell enterprise-grade tech support for this engine and is working out the pricing right now with the initial customers who have it in production. This will have per-host pricing, says Johnston, but the pricing will not be set for around a month as Docker talks to these early adopters and gets feedback on pricing models. The host pricing will include an unlimited number of containers per server and an additional fee for 24x7 support.

Docker also plans to commercialize the Docker Hub, which is a repository for pushing and pulling software images onto hosts as they are created for development, test, and production environments. The public software repositories will be free, explains Johnston, but if you want a private repository for applications, then the fee is going to be $7 per month for up to five images.

dotCloud was founded in 2008 by Solomon Hykes and has itself raised $28.7 million in venture funding to get its platform cloud, which is based on Docker, off the ground and competing against giants. The company changed its name to Docker earlier this year and shifted its emphasis from building a cloud to commercializing the Docker container technology. Concurrent with this change, Docker raised a $15 million round of Series B funding back in January led by Greylock Partners and with Insight Venture Partners also coming in. Benchmark, Trinity Ventures, and Jerry Yang/AME Cloud Ventures, who all participated in an undisclosed Series A round, also kicked in money on the second go around. So the original investors like the new strategy. Importantly, Jerry Chen, who was in charge of cloud and application services at VMware, scaling it up from 400 to 15,000 employees and up to $5 billion in revenues, is the Greylock partner that is sitting on the Docker board. Greylock was an early investor in Red Hat and Cloudera as well as in LinkedIn, Facebook, and Palo Alto Networks, just to drop a few more names.

The dotCloud platform cloud still exists, but the odds are that others will build similar Docker-based clouds and therefore provide some kind of portability between clouds – for Linux applications, at least. This is something that is very difficult to do down at the infrastructure layer of clouds because of the differences in hypervisors, packages, and APIs in various clouds.

Johnston can't name names, but says that all of the big financial services institutions are looking at Docker, and says that Baidu, the largest search engine in China, and Yandex, the largest in Russia, are both spinning up and taking down hundreds of thousands of Docker containers each day on their platform clouds. eBay uses Docker as well. Google, of course, has its own containerization that is similar to Docker, called lmctfy. Google recently said that it fires up over 2 billion software containers a week on its vast fleet of servers, estimated to be well over 1 million machines.

Docker Engine is supported on Red Hat, Fedora, SUSE Linux, Debian, Ubuntu, Gentoo, and Arch Linux distributions. The software has over 2.75 million downloads and over 14,000 applications have been "Dockerized" and published in the Docker Hub, and they are all of the big names in the open source hyperscale software stack as you might imagine. There are over 6,500 projects on GitHub related to Docker, just to give you an idea of how other software projects are looking to use Docker as a shrink-box wrapper for code.

This, of course, is what VMware wanted nearly a decade ago for its ESXi server virtualization hypervisor, and to a large extent in the enterprise, an ESXi virtual machine has become the de factor box in which software is poured. But for many industries, Linux is the preferred platform and it is not hard to see that given the flexibility and portability of applications that Docker is providing, it will not be long before Docker rises to the top of the stack for applications that need only containers instead of full server virtualization.

It will be interesting to see how Docker deals with Windows over the long haul – if it even bothers. While Linux has its share of workloads in the enterprise, Windows still dominates for a lot of applications, and Windows could use a lightweight container technology to complement Hyper-V and ESXi, which rule on virtualized Windows platforms. Microsoft would be wise to have a software container of its own. There is a way, of course, to plunk Docker containers down on top of Hyper-V and ESXi virtual machines. But this lacks the simplicity of the Linux approach.