So far, we've talked about starting up individual containers or networking different types of containers to each other. But of course, one of the promised benefits of running your app on containers is that you can quickly scale out a single container by running multiple instances.

Doing this manually isn't really an option. You'd have to have many people constantly monitoring your web traffic, spinning up and down containers. No, you need to be able to do this automatically. This is where orchestration software comes in. In the world of containers, orchestration refers to managing a large set of containers, including horizontal scaling.

Orchestration software takes care of spinning up and down individual containers while monitoring the health of the containers, all across multiple physical or virtual servers. Generally, you define what your app should look like in an abstract way using YAML or JSON. From there, the orchestration software takes care of the underlying details of managing containers and the underlying physical resources they are running on. This splits apart the roles of architecting an app and architecting the hardware the app is running on.

For the devOps role, the hardware is just one deployment target that the orchestration software will handle. As Apache Mesos promises, you can program against your data center like it's a single pool of resources. There are generally two types of orchestration tools: those designed to run in your own data centers with you managing them and services for orchestrating in the cloud.

Many, but not all, of the cloud services are built on top of local tools, meaning that this distinction isn't perfect, but for our purposes, it will do. Let's look at the on-prem orchestration tools first. The most important to be aware of are Kubernetes, Mesosphere Marathon, and Docker Swarm. Kubernetes is a Google project and is one of the oldest and most popular orchestration tools, although that doesn't make it very old in the grand scheme of things.

Google uses a version of Kubernetes internally to manage their production apps. Kubernetes is now largely focused on orchestrating Docker containers, although it was originally built for Google's internal pre-Docker container technology, and can also manage Rocket containers. Kubernetes works with a master node paradigm. Each cluster is made up of a master server that runs Kubernetes and a number of node servers, which Kubernetes manages. Your container runs on the node servers, and Kubernetes manages which servers are running which specific containers. Next is Marathon for Apache Mesosphere. Mesosphere is touted as a data center operating system, and Mesos is its underlying kernel. Mesosphere lets you treat your entire data center as a single entity. You run processes on Mesosphere, and Mesos decides which physical components to put towards that job without you having to think about it.

Marathon is the software package for Mesosphere that specifically handles container orchestration. Some very large companies, such as eBay, Verizon, and Twitter, really on Mesosphere. Finally, Docker has built orchestration technology directly into the core of its engine with its latest 1.12 release, which they call Swarm. This is a little confusing, because Docker Swarm used to be a standalone product, and the new standard Swarm mode is a set of features built right into the Docker engine and is slightly different than the old Swarm.

Swarm positions itself as being more lightweight and easy to use than Kubernetes or Mesos. It is a decentralized orchestration system where each node can manage itself and its own traffic instead of being managed by a central master node. In addition to these major on-prem orchestration tools, there are a number of cloud platforms that have container orchestration built in, some built on top of these tools and some offering their own interfaces.

For instance, AWS offers its EC2 Container Service, or ECS, which will manage a cluster of EC2 instances with Docker running on top of them. Rather than using a specific container orchestration system, ECS is built with common AWS tools like Elastic Load Balancer and IAM. Azure also offers a container service, ACS.

Google also offers container hosting and management with its Google Container Engine, or GKE system. As you might expect, GKE uses Kubernetes. Docker offers Docker Cloud, its own cloud container service. Docker Cloud uses Docker Swarm to manage hosted applications and also lets you perform other Docker-specific tasks in the cloud, such as image management and CICD and lets you manage your cloud account from Docker's desktop tools.

Docker Cloud is cloud hosting built from the ground-up for containers, rather than being container hosting added to an existing cloud hosting service. So that should give you an idea of what orchestration is and the main tools people use. The orchestration ecosystem is still new, and new tools pop up frequently, but this should give you an understanding of what these tools are and how they work.