TNS Research: The Present State of Container Orchestration
After watching a year’s worth of conference presentations, it is easy to become inured by the hype around products that orchestrate containers. Yet, there still confusion in the broader tech community about what functionality is involved with orchestrating containers. The New Stack ran a survey that was able to target users of container management technology, with 70 percent of respondents using containers to some degree. We found that there is indeed uncertainty about what it means to manage and orchestrate containers. There is also a developing consensus that scheduling, cluster management and service discovery are necessary to orchestrate the use of containers in production.
Over 40 percent of respondents using or trialing containers said an orchestration platform was the primary way containers are managed in their organization. However, the actual products being used or planned for use were strongly influenced by a respondent’s stage of container adoption. People that have non-production use of containers were more likely to cite configuration management tools like Ansible and Chef as a method of orchestration. Platforms-as-a-service (PaaS) solutions such as Red Hat OpenShift and Hashicorp products were cited more often in their roadmaps among those evaluating containers.
There is a wide mix of companies and services in users’ plans. It remains to be seen what tools will win out. For now, it appears that many of the container orchestration-specific tools, like Kubernetes, Mesos and Docker Swarm, as well as the Containers-as-a-Service offerings, are making sure that they provide a wide range of open source tools within their bundled offerings. Vendors should realize that most users are not looking for one solution to solve all their needs — only 29 percent said it is important for a container orchestration tool to be able to handle non-containerized and legacy applications. Instead, what they care most about it is integrating tooling for both developers and IT operations (ops).
Sample and Methodology
How the Data Was Collected
The survey was administered via the web from February 21 through March 14, 2016. As with all anonymous web-based surveys, the results are likely affected by self-selection bias. All responses were manually reviewed for integrity, and several were discarded. The data in this chapter is based on 309 respondents, 75 percent of which completed the entire survey.
[cycloneslider id=”ebook-3-sponsors”]
The New Stack readership was asked to participate in the survey. The study was also promoted via social media. Also, we received 54 responses due to publicity in DevOps Weekly and another 42 as a result of an email sent by an executive in Mesosphere’s marketing department. When relevant, the analysis notes how this may impact the findings.
Profile of Study Participants
Sixty-one percent of respondents were end users, with the remainder working for vendors and service providers associated with containers, PaaS, infrastructure as a service (IaaS) or software management. Questions about tools or products being used or considered were only asked of end users to prevent vendors from voting for their products. Furthermore, these questions were only presented to the 164 end users that were using and/or at least trialing or evaluating containers.
Compared to the broader market, the survey was more likely to talk to early adopters, as 59 percent said they use containers in production, with another 11 percent using containers in a more limited manner, such as in the test part of the development lifecycle. It is somewhat surprising that only 64 percent of vendors said they are using containers in production.
Almost half of the end users described themselves as DevOps, and upon investigation, we found this was not because of responses received from DevOps Weekly subscribers. Although the growth of DevOps is real, the role is mostly likely strongly represented in this study because container orchestration is an area that requires coordination between application development and IT operations.
Defining Container Orchestration Functionality
When we started writing our eBook on container automation and orchestration, there was little agreement about what exactly defines container orchestration. So, instead of trying to create a top-down definition of the market, we decided to ask the community about their opinion.
There is a consensus that scheduling, cluster management and service discovery are core elements. Four out of five people said scheduling and cluster management are expected in a product or service related to container orchestration. Close behind, 76 percent associated service discovery with container orchestration. Monitoring, and especially configuration management, were the least likely functionality associated with container orchestration. Auto-scaling, load balancing and networking were all definitions that were proffered by multiple people in the “Other” box.
A couple of variables affected perceptions, with job role having a significant impact on how people define container orchestration. In particular, self-identified application developers were much less likely to associate it with scheduling (59 percent) and more likely to associate it with monitoring (64 percent) and configuration management (64 percent). Their lack of involvement with monitoring and configuration management may be why they cited the functionality more often. In addition to job role, we also found that the people with a Mesosphere perspective – those that participated in the survey due to a company employee’s promotional email – were more likely to answer scheduling (93 percent) and monitoring (74 percent). Among those Mesosphere-centric respondents, 72 percent said they use Apache Marathon for scheduling.
Defining Containers as a Service Functionality
Like PaaS before it, there is considerable disagreement about what Containers as a Service (CaaS) means. It is important to note that The New Stack asked about some, but not all the types of CaaS functions that can be found in the market. We found that container orchestration was the functionality most associated with Containers as a Service. However, since the survey’s topic was container orchestration, it is likely that the responses were biased towards that response. Registry, the place where containerized content is accessed, was also strongly correlated with CaaS. Building images and runtime were least likely to be associated with CaaS, yet still over 50 percent said they expected those abilities to be in an offering. In other words, CaaS is viewed by a majority as an end-to-end offering.
Job role also affected understanding about CaaS. Compared to DevOps, people responsible for IT operations were less than likely to say registry (61 vs. 81 percent), and to a lesser degree runtime and container orchestration, are part of a CaaS package. Those responsible for IT operations were less likely to be using containers, so they were probably less informed about why registries can be an important part of a container service. Possibly because they are not as tied into the container ecosystem, participants solicited from the DevOps Weekly newsletter were less likely to say container orchestration and runtime are part of CaaS.
Outside of Vendor Territory, Confusion Abounds
Employees of companies associated with the container ecosystem were much less likely to check most of the boxes when asked to define functionality associated with either container orchestration or CaaS. In particular, fewer vendors said cluster management equates to container orchestration, and that registry, deployment and cloud orchestration are components of CaaS.
[cycloneslider id=”ebook-3-sponsors”]
One reason employees of vendors involved with containers are more discriminating in their definitions is that they have likely spent more time attending events and reading about the subject. Another reason may be that a broader definition of the market does not fit into how they are marketing their products. No matter why the differences arise, they point to the fact that beyond industry insiders, there is a continuing need to define the market.
Products/Services Used to Manage and Orchestrate Containers
As noted in several of our previous articles, no matter how you slice it, the use of containers is rising and with it the need to orchestrate their deployment in production. Since there is still confusion about what container orchestration means, we made sure to ask broadly about what is being used to either manage or orchestrate containers. The questionnaire first asked about the primary method of orchestration or management, and then asked about the top three products or services to be used in the next year. This methodology forced respondents to prioritize between multiple methods and offer some specificity about their future roadmaps.
One consequence of this approach is that only 16 percent of respondents primarily manage containers with shell scripts or customizations, which is significantly lower than other studies where respondents were able to provide multiple choices. While customizations may glue together multiple tools, they are usually not in and of themselves the main way to manage something.
[cycloneslider id=”ebook-3-sponsors”]
Status of container adoption also correlates with the primary way of orchestrating containers. Respondents that have started to use containers, but not yet in production, were a bit more likely to use configuration management tools (21 percent) and shell scripts (21 percent). However, as users move forward on their “container journey,” the numbers drop to 16 and 10 percent respectively, possibly because the need for a more consistent, scalable solution becomes apparent as production use increases. Note that subscribers to DevOps Weekly were more likely to be using configuration management tools (29 percent), which is not surprising considering its publisher’s day job is with Puppet Labs.
Looking at PaaS and CaaS, it appears that PaaS is more likely (13 percent) to be top-of-mind among those conducting evaluations. Production users are more likely to say CaaS is their primary management method. Of course, as we discussed earlier, the definition of Containers as a Service is still up for debate. Some people may consider it to be the provision of container services or container hosting, while others like Docker have a different definition, focusing more on technology-agnostic portability.
Despite all these other methods of managing containers, the leading choice by far was orchestration platforms like Swarm, Kubernetes and Mesos, which were cited by 45 percent of respondents as their primary method of managing containers. Also described as a framework, orchestration platforms offer users the ability to execute several types of orchestration functionality (i.e., cluster management, scheduling). At least for now, it appears that purpose-built platforms are the orchestration method of choice, but that may be due to the field’s newness rather than because of a preference for one type of tool or another.
IT operations is a job role that doesn’t seem as inclined to use orchestration platforms — fewer than half as many cited them as compared to everyone else. Instead, 35 percent of IT ops primarily use configuration management tools, compared to only nine percent among everyone else. Also, IT operations are just as likely to look towards CaaS as to orchestration platforms. In comparison to those with an operations focus, respondents with application development roles were more likely (27 percent) to use shell scripts, which is not surprising since they are likely very comfortable hacking a custom solution together.
The names behind the categories reported in the charts were revealed by asking about the top three products in users’ near-term plans as well as those currently being used. The top five choices were Kubernetes, Ansible, Mesos or Mesosphere DCOS, Amazon Elastic Container Service (ECS) and Docker Swarm. Kubernetes leads the list with 39 percent of respondents planning to use it. However, since it is not a purchasable product, it is hard to compare that result with the other entries in Figure 10. Another way to read the chart would be to identify products like Google Container Engine (GCE) and CoreOS Tectonic that are closely related to Kubernetes. Since many users of Amazon ECS also use Kubernetes, it is hard to determine how many of the 25 percent citing Amazon Web Services (AWS) plan to use it as their primary method of orchestration. The same issue arises with Mesosphere DCOS, which can be run on top of Amazon EC2.
One out of three respondents expects to use either Apache Mesos or Mesosphere DCOS. While the two are different, they were combined to account for respondents that think of them as synonymous. The sample likely over-represents plans to use Mesos because almost fifty percent of respondents citing Mesos or Mesosphere DCOS were solicited for participation by Mesosphere. However, even if those respondents are excluded, Mesos/Mesosphere would still tie for fourth place at 23 percent.
The use of Docker for orchestration is understated if you only look at the 23 percent figure for Swarm. As stated earlier, the production-ready version of Swarm was announced while the survey was being conducted, but so was a new product called Docker Datacenter. Furthermore, Tutum was renamed as Docker Cloud as a production version went live. There was no overlap between people citing Docker Datacenter and Docker Cloud, but seven of those 15 respondents did cite Swarm as another of their choices. Thus, looking at Docker more broadly, 31 percent plan to use the company’s products to manage containers.
Ansible was the choice of 36 percent of respondents, which surprisingly placed it second on users’ roadmaps. Among the Ansible respondents, only ten percent chose configuration management as their primary method of orchestrating containers. It appears that Ansible will be used in conjunction with several other tools. Since Puppet Labs and Chef are also configuration management vendors, it is not surprising to see that all three are much more likely to be in the roadmaps of the respondents that are using containers but not in production. However, since Ansible is still number three among those using containers in production, we believe there are other reasons why Ansible does so well. Chief among them is its recent purchase by Red Hat. Already well-regarded, it is possible that many Red Hat customers will choose Ansible over its configuration management rivals as a way to integrate with other Red Hat offerings. This rationale may be the reason that seven of the twelve respondents citing Red Hat’s OpenShift also plan to use Ansible.
Among the second tier of choices, people conducting trial projects or evaluations were more likely choose Red Hat’s OpenShift and Hashicorp products. This is likely because many end users are already familiar with or using those companies’ products. For example, the percentage planning to use a Hashicorp product rose from 13 percent to 21 percent when looking only at people that use Hashicorp’s Consul for service discovery. Since Hashicorp’s Nomad is meant for larger deployments, its use may go up as the scale of current container usage increases.
Tools Being Used to Perform Specific Functionality
Warning: The following charts and analysis should not be used to analyze market share. Instead, view them as a gauge of user perceptions. Survey participants were offered a long list of tools that could conceivably be related to container management functionality. Although they were able to offer other, open-ended responses as well, this methodology may have influenced the results. Furthermore, no definition of what service discovery, scheduling or cluster management was provided.
Service Discovery
Consul from Hashicorp, Apache ZooKeeper and etcd were cited most often for service discovery. Since Mesos utilizes ZooKeeper to ensure high availability, it is not surprising that 73 percent of participants solicited through the Mesosphere promotion of the survey use ZooKeeper for service discovery. Those that heard about the survey in DevOps Weekly were more likely to use Consul or etcd, perhaps because they are attracted to open source solutions.
Scheduling
Kubernetes, Marathon and Docker Swarm were the tools cited most often for container scheduling. Among the Mesosphere-solicited respondents, 72 percent use Marathon for scheduling and 33 percent use Chronos, while only six percent use Kubernetes. If these responses are excluded, Marathon would trade places with Swarm, dropping from the second to the third most used tool.
Interestingly, this group was only slightly more likely to use Mesosphere DCOS for scheduling, perhaps because they are not as interested in the enterprise-level services that DCOS recently brought to the market. On the other hand, 43 percent of participating DevOps Weekly subscribers use DCOS for scheduling. It is notable that both AWS ECS, Rancher, and “homegrown” each received at least four write-in votes.
Cluster Management
Mesos, Kubernetes and Docker Swarm were the tools cited most often for cluster management. Every participant that came from Mesosphere said they use Mesos. Yet, even if we exclude these answers, Mesos would still compete with the other two with 22 percent of the remaining sample. DevOps Weekly subscribers were more likely to use Kubernetes (50 percent) and Docker Swarm (43 percent), which may be an indicator of these tools’ exposure outside of the early adopter community. Since Swarm moved into general availability in November 2015, it is not surprising that half of Swarm respondents are not using containers in production. Docker’s Universal Control Plane (UCP) is also used by five percent of respondents, of which all but one person were using it in conjunction with Swarm to manage clusters of containers.
Evaluation Criteria
In this still nascent field, deciding what product or service to use is not as easy as just looking at cost, performance tests and brand reputation. To find out how container orchestration decisions are being made, we asked about users’ evaluation criteria regarding requirements and abilities.
We admired many of the questions Docker asked in its recent user survey of their users, so we adapted one for our own survey. Just like in Docker’s results, The New Stack found that integrated tooling (for both developers and IT operations) and addressing the full application lifecycle were among the most likely criteria to be considered important. Also in alignment with the Docker survey, we found that supporting multiple operating systems is the least important. Yet, looking more closely at Figure 16, it becomes apparent that addressing the needs of specific job roles is essential. Fifty-eight percent said integrated tools for both application development and IT operations is extremely important, 12 points higher than any other criteria.
Usability in many different environments is something that a technology can improve on as it matures. It is harder to address the ability to handle specific use cases. Figure 17 indicates how critical being able to support long-running applications is, with 86 percent saying it is at least very important. In contrast, only 43 percent said the same thing about batch applications. Load balancing was also considered important by more than three-quarters of participants. Persistent storage, something that is often cited as a pain point, was as at least very important for 70 percent. However, in comparison to the leading criteria, people were less likely to say it is extremely important. At the bottom of the list is being able to support non-containerized workloads, which indicates that people expect to handle legacy applications separately from those running on containers.
Takeaways
Definitions of Container Orchestration
- Scheduling, cluster management and service discovery are widely acknowledged to be part of container orchestration. However, provisioning and monitoring were also considered to be part of orchestration by over half of respondents.
- Docker’s Swarm is recognized as the underlying technology among many of those planning to use Docker Cloud and Docker Datacenter.
- While there is still work to be done, many users of Mesos and Kubernetes understand what products use the underlying technology.
- Container orchestration platforms remain the most commonly used method of managing containers. However, when looking at specific offerings, users are most likely to say they use Kubernetes which, in and of itself, is not a product.
Containers as a Service and Platform as a Service
- There is agreement that CaaS encapsulates several parts of the software development lifecycle. Container orchestration and registries are the abilities most associated with the term.
- At 25 percent, AWS Elastic Container Service is the fourth most considered option for managing containers in the next year. Google Container Engine and Docker’s Cloud and Datacenter offerings also are often cited.
- There is still not enough data to determine how much bundling users want in their orchestration tools.
Job Roles
- The intersection between developers and IT management continues to be the container sweet spot. Considering that 49 percent of respondents described their job role as DevOps, it is not surprising that integrated tooling for both developers and IT operations was considered extremely important by 58 percent.
- Application developers are more likely to use custom scripts to manage containers. Perhaps because they do not use them a lot, this group is more likely to think monitoring and configuration management are part of container orchestration tools.
- IT operations are more likely to be using configuration management tools to handle non-production container implementation. There are indications that they will be more likely than other groups to look at CaaS for production use instead of more generic “orchestration platform” offerings.
Configuration Management
- Configuration management was the functionality least likely to be associated with container orchestration, although the big configuration management vendors are still often cited in users’ plans.
- Instead of being a primary method of orchestration, configuration management vendors can expect to be used in conjunction with other tools. Thus, Ansible, Puppet Labs, Chef and SaltStack all get cited in many users’ roadmaps as a second or third choice.
- Ansible rivaled Kubernetes for a number of citations in users’ plans. Red Hat’s acquisition of Ansible may have boosted its position as many people cited both OpenShift and Ansible in their plans.
