DockerCon 2016: Docker’s Plan to Dominate the Enterprise Data Center
Last week Seattle hosted DockerCon 2016, industry’s largest technology conference focused on containers. Like most of the vendors, Docker has utilized its user conference to announce a slew of new products, features, and services. These announcements reveal Docker’s ambitious plan to make containers the basic building block of the enterprise data center.
Docker Engine’s Built-in Orchestration
The latest beta of Docker Engine comes with built-in support for Swarm, Docker’s official container orchestration engine. This eliminates the need for installing, configuring, and managing a separate cluster to run microservices and distributed containerized workloads. While preserving backward compatibility, the core engine and tools got additional capabilities to run containers across a fleet of servers. There is a new nomenclature of application bundles, nodes, and services to manage containerized applications. In its latest avatar, Swarm comes with most-requested features like service discovery, load balancing, and resiliency, closing the gap with some of the competing orchestration engines in the market.
This seemingly routine integration has many implications. Containers running in production typically span more than one machine. Container Orchestration Engines (CoEs) enable customers to manage the workloads as well as the underlying infrastructure. The bottom line is that the orchestration is key to run production workloads. Customers have a choice of CoEs — Kubernetes, Docker Swarm, and Mesosphere DC/OS — to manage container infrastructure.
Demo: @Docker engine automatically restarts failed containers #Dickerson pic.twitter.com/jERfaFlYzo
— The New Stack (@thenewstack) June 20, 2016
The interesting fact is that all the available container orchestration engines are built for Docker engine. Though they can work with other container technologies, they are primarily designed to support the image format, API, and runtime that Docker exposes. As an early mover, Docker has already won the battle at the core engine level. But the real opportunity exists in managing the distributed workloads rather than one container at a time. Docker Swarm, Kubernetes, and Mesosphere are battling it out to seize this opportunity (among others).
While it’s important to get the developer mindshare, containerization is predominantly an operations-oriented technology.
Configuring and deploying containerized workloads on the new Swarm environment is a no brainer. With just a few commands, developers and operators will be able to use the built-in orchestration capabilities than dealing with the complex setup process involved in configuring Kubernetes or Marathon.
Customers who start using Swarm mode of Docker 1.12 may prefer Docker, Inc.’s commercial offering called Docker Datacenter, which comes with a private registry, management layer, and a robust Docker engine optimized for production workloads. So, the game plan is clear – Docker Swarm is for dev/test and Docker Datacenter for production. This move not only impacts the competition but also the overall container ecosystem.
As its traffic spiked, @Zenly used @Docker Engine 1.12 to simplify its orchestration and monitoring stack #dockercon pic.twitter.com/eEZ1Z3G4Sc
— The New Stack (@thenewstack) June 20, 2016
Docker has redefined DevOps by enabling Ops teams to standardize the unit of packaging and deployment. According to the latest Artifactory survey, analyzed by The New Stack, 80 percent of users voted Docker as the chosen format for packaging software.
Irrespective of the language, runtime, and deployment target, software can be packaged as Docker images. Dockerfile, the declarative format for describing an image, is used extensively in defining build pipelines.
To describe multiple, dependent container images deployed together; DevOps teams are using Docker Compose file. Most of the contemporary software stack is available in one of these formats. Bitnami has started to offer Docker images along with its traditional line of prepackaged VMs. After Facebook open sourced Parse, many organizations have been relying on Docker to deploy the popular mobile-backend-as-a-service (MBaaS) stack across multiple environments.
.@Docker intros (a still experimental) portable format for multi-container apps, DAB: https://t.co/nMSLE09clH @solomonstre #Dockercon
— The New Stack (@thenewstack) June 20, 2016
Docker has announced a new format called Distributed Application Bundles (DAB) to make it easy to deploy multi-container applications. DABs are more elegant and efficient when compared to Docker Compose files. They produce consistent results each time the stack is deployed, which aligns with the philosophy of “Infrastructure as Code.”. When integrated with CI tools such as Jenkins, DAB enables a unified way to package all the artifacts required by a complex, distributed application. Docker is taking a leaf from AWS CloudFormation playbook to define a specification that is cleaner and simpler than the existing specs.
Docker on Mac, Windows, AWS, and Azure
With application lifecycle management gradually shifting to containers, Docker is marginalizing the role of the operating system. Once Docker engine is installed, the OS is reduced to a thin layer with very limited responsibility. This shift has resulted in lean-and-mean operating systems such as Alpine, CoreOS, Atomic Hosts, Snappy Core, RancherOS that are optimized to run containers. When deployed on public cloud environments, these new breeds of operating systems combined with Docker bring web-scale to applications.
#Docker for Mac & Windows beta is now public: http:// https://t.co/QGv1sU2XNO @solomonstre #dockercon
— The New Stack (@thenewstack) June 20, 2016
With the latest push of native installers for Mac and Windows, Docker is invading the developer workstation. Docker on desktops will alleviate the pain involved in configuring and reconfiguring development environments for disparate languages, runtimes, frameworks, and libraries. Code editors and IDEs can be natively integrated with Docker engine for seamless development and test workflows.
In a large enterprise, it is common for each department to rely on an environment that meets their requirements. LAMP, Java, and .NET are the predominant platforms used in enterprises. Enterprise IT can publish standardized container images to the Docker Trusted Registry to facilitate consistent environment setup across the organization. This enforces better governance and consistent application development and deployment policy.
Docker launches "Docker for AWS" and "Docker for Azure" betas, for native Docker Engine 1.12 integration @solomonstre #DockerCon
— The New Stack (@thenewstack) June 20, 2016
Docker is also making it easy to configure its commercial Containers as a Service (CaaS) offering on the most preferred enterprise public cloud platforms — AWS and Azure. It has published AWS CloudFormation and Azure Resource Management templates to enable one-click deployment of Docker Datacenter.
Docker on development machines, Swarm in test and staging environments, and Docker Datacenter in the production environments bring the continuum that aligns with Docker’s vision of Build, Ship and Run.
Irrespective of the stack, environment, and tools, Docker is marching towards becoming the de facto standard for software development. Native integration with Mac and Windows will be a big leap in that direction.
Docker Store – The Container Marketplace
.@Docker launches Docker Store, a marketplace of scanned free & commercial software, now in private beta https://t.co/ijjE7jWVLz #Dockercon
— The New Stack (@thenewstack) June 21, 2016
As mainstream applications move towards containerization, the ecosystem will be flooded with apps. This may impact the quality of packaged applications.
Docker Store is a marketplace for free, open source, and commercial software packaged in a containerized form. It will be Docker’s authoritative source for users to search and install software. For independent software vendors, Docker Store will be a destination to publish their applications. Docker will maintain the marketplace by ensuring that the published software meets the security standards and best practices of containerized software.
Docker Store is Docker’s attempt to stake a bigger claim in the container ecosystem. Docker hopes that a well-run app store will turn the company into the custodian of the future applications.
Necessary and Sufficient?
Docker certainly has an early mover advantage in the container market. It also scores high on the simplicity factor. No doubt, Docker Swarm makes it a breeze to deploy distributed containerized applications. But the relative lack of community participation is one of the glaringly visible gaps in Swarm. When compared to Kubernetes, Swarm comes across as a tool that is primarily driven by Docker.
Though Google continues to be the top contributor of Kubernetes, Red Hat, CoreOS, Huawei, and many other companies contribute to the effort as well. With a vibrant community and the support of a broader ecosystem, Kubernetes is aiming to be a capable CaaS platform for enterprises.
Apps always betray their origins. If an app was built by a company w 30k engineers, it will be pretty specific for that use case–@golubbe
— The New Stack (@thenewstack) June 20, 2016
Docker claims that using Docker Datacenter offers ultimate portability with no vendor lock-in. That’s not a sufficiently compelling reason for enterprises to choose Swarm. Once the software is packaged as a set of container images, it becomes portable. The deployment mechanism and terminology may differ, but as long as the lowest common denominator remains as the image, applications are portable.
The format of the deployment descriptors such as Docker Compose files, Kubernetes deployment definitions, and DC/OS Marathon application configuration differ from each other. But that’s only the metadata to describe a deployment. A thoroughly designed and tested microservices application would seamlessly run on any orchestration engine deployed on physical or virtual infrastructure.
Over 460,000 apps have been Dockerized, 4.1 billion pulls from #DockerHub — @Golubbe #dockercon pic.twitter.com/PEs7cuOM1I
— The New Stack (@thenewstack) June 20, 2016
The strategy of bundling doesn’t influence enterprise decision makers in all cases. If that were the case, almost every enterprise running a JVM would have embraced Oracle’s application server. The enterprise Java application server market is dominated by IBM WebSphere and Red Hat JBoss. Docker Swarm might offer a low-friction clustering environment to developers, but for the production environment, customers would choose a platform that’s aligned with their business requirements.
There is also a perception among customers that Docker is more aligned with developers than the operators. While it’s important to get the developer mindshare, containerization is predominantly an operations-oriented technology.
Traditional DevOps companies like HashiCorp and Chef are expanding their portfolio to support containers. The recent announcement of Habitat from Chef is a classic example of that trend. HashiCorp’s Nomad plays well with the ecosystem of Vagrant, Packer, Terraform, and Otto.
Docker has to increase its focus on the ops part of DevOps by making it easier for operators to integrate containers with their existing toolchain.
"The problem with orchestration is that the technology is there but it is only usable by experts."-@solomonstre #dockercon
— The New Stack (@thenewstack) June 20, 2016
Docker is eager to move up in the infrastructure stack going after the low hanging fruit of container management. There are multiple gaps that need to be closed before containers become mainstream in enterprises. Storage, networking, and security are critical that needs attention. Similar to the virtualization market, containerization will be an ecosystem play. Traditional vendors with a proven track record have just started to pay attention to the containerization trend. Their contribution would be critical to driving the next wave of infrastructure in the enterprise data center.
Heterogeneity is not new for Enterprise customers. They have dealt with Microsoft .NET applications talking to DB2 databases running on mainframes. Customers will always choose the best of the breed tools that are designed to deliver the results. The container market is not an exception to that. Get ready to deploy applications on a stack that’s running a dozen infrastructure technologies, including Docker.
May the best platform win!
Bitnami, CoreOS, Docker, IBM, Mesosphere, and Red Hat are sponsors of The New Stack.
Feature image: Docker CEO Ben Golub, Dockercon 2016