Docker has made great strides in advancing development and operational agility, portability, and cost savings by leveraging containers. You can see a lot of benefits even when you use a single Docker host. But when container applications reach a certain level of complexity or scale, you need to make use of several machines. Container orchestration products and tools allow you to manage multiple container hosts in concert. Docker swarm mode is one such tool. In this course, we’ll explain the architecture of Docker swarm mode, and go through lots of demos to perfect your swarm mode skills.
After completing this course, you will be able to:
- Describe what Docker swarm mode can accomplish.
- Explain the architecture of a swarm mode cluster.
- Use the Docker CLI to manage nodes in a swarm mode cluster.
- Use the Docker CLI to manage services in a swarm mode cluster.
- Deploy multi-service applications to a swarm using stacks.
This course is for anyone that is interested in orchestrating distributed systems at any scale. This includes:
- DevOps Engineers
- Site Reliability Engineers
- Cloud Engineers
- Software Engineers
This is an intermediate level course that assumes:
- You have experience working with Docker and Docker Compose
Welcome to this overview lesson on Docker swarm mode. We'll get a conceptual understanding of swarm mode in this lesson before understanding its architecture and diving into the details and demos in following lessons.
We'll start the lesson by getting an understanding of why we need swarm mode. After that, we'll highlight some features of Docker swarm mode to understand what swarm mode can do for you. Next, we'll learn about the main concepts of Docker swarm mode. Lastly, I'll touch on the universal control plane which is Docker's enterprise product built on top of swarm mode.
Docker has made great strides in advancing development and operational agility, portability, and cost savings by leveraging containers. You can see a lot of benefits even when you use a single Docker host. But when container applications reach a certain level of complexity or scale you need to make use of several machines. Container orchestration products and tools allow you to manage multiple container hosts in concert. Docker swarm mode is one such tool.
Swarm mode is a feature built into the Docker Engine providing native container orchestration in Docker. Swarm mode is something you need to enable and when you do, the Docker Engine is said to be running in swarm mode. With swarm mode you can control a cluster of machines in a way that is similar to running and about as easy as running a single Docker Engine. Of course there are some differences and we'll see them in this course.
Calling swarm mode a container orchestration feature doesn't quite do it justice. It encompasses cluster management, container orchestration, and more. Some of the main features of swarm mode include:
" Integrated cluster management within the Docker Engine without any additional software
" A declarative service model that allows you to declare what you want and Docker can create it for you. There is no need for you to specify the sequence of commands to realize what you want.
" Swarm mode is able to monitor the cluster state and reconcile any differences between the desired state and the actual state. (Desired state reconciliation)
" Swarm mode uses certificates and cryptographic tokens to secure the cluster
" As well as features you'd expect in a container orchestration offering such as service scaling, multi-host networking, resource-aware scheduling, load balancing, rolling updates, restart policies, and more.
Docker actually has two cluster management solutions. Both are open source and live on GitHub. Surprisingly, they are both called swarm. Docker Swarm, with a capital S, was the first container orchestration project by Docker. It uses the Docker API to turn a pool of Docker hosts into a single, virtual Docker host using a proxy system. To reduce confusion, Docker Swarm is now referred to as Docker Swarm standalone in documentation.
Although Docker Swarm standalone project is still maintained, the newer container orchestration tool is called Swarmkit. It is what is built into the Docker Engine since Docker version 1.12. You might see swarmkit mentioned from time to time, but this is the most commonly referred to as swarm mode. Docker recommends Swarm mode unless you have a specific reason to use Swarm standalone.
So now you know that there are two swarms, Swarm standalone and swarm mode. It's useful to be aware of the distinction. You might search for Docker swarm online and stumble upon something related to Swarm standalone when you wanted swarm mode. To avoid any confusion, this course deals exclusively with swarm mode. In the remainder of the course, if I refer to swarm, I'm referring to Docker running in swarm mode. In practice, it's pretty common to drop mode from the name although it can potentially lead to misunderstandings. In the remainder of the lesson, we'll cover the architecture of swarm mode.
Swarm Mode Concepts
Before going too far, we'll cover some of the main concepts and swarm mode terminology.
A swarm consists of one or more Docker Engines running in swarm mode. Each instance of the Docker Engine in the swarm is referred to as a node. It is possible to run multiple nodes on a single machine. For example, by using virtual machines. In production environments, you should use multiple machines to ensure availability of the swarm if a machine goes down.
Nodes can participate in a swarm by taking on specific roles: managers and workers. Every swarm requires at least one manager. Managers have several responsibilities, but we'll start simple and consider one main responsibility. Managers accept specifications from users and drive the actual state of the swarm to the specified desired state. They do so by delegating units of work to workers in the swarm. Workers are primarily responsible for running the delegated units of work. Workers also run an agent which reports back to managers on the status of their work. A node can be either a manager, or a worker.
The specifications that users submit to managers are called services. This is the same concept as a service in Docker Compose. The service configuration declares its desired state, which includes the networks and volumes it uses, the number of replicas, resource constraints, and other details. A manager will ensure the actual state of the swarm matches the service configuration. if it is possible to realize in the swarm. There may not be enough available resources in the swarm which would prevent the desired state from being achieved. Docker will also make the changes necessary to reconcile the actual state with the desired state if you update a service.
There are two kinds of services: replicated and global. You specify the number of replicas for a replicated service based on the scale you desire. A global service allocates one unit of work for each node in the swarm. Global services can be useful for monitoring services, for example.
The units of work delegated by managers to realize a service configuration are referred to as tasks. The tasks correspond to running containers that are replicas of the service. Managers schedule the tasks across nodes in the swarm. If a node leaves the swarm, the tasks that the node was running will be scheduled onto the remaining nodes in the swarm.
By default, manager nodes also run tasks like workers. You can configure managers to participate exclusively in managing the cluster and that is probably a good idea in production. Allowing managers to run tasks by default enables easy to setup and functional single node swarms.
Universal Control Plane
The last topic I want to cover in giving an overview of swarm mode is the Universal Control Plane (UCP). UCP is only relevant for the enterprise edition of Docker so I will only briefly touch on it.
Working with swarm mode is similar to working with Docker. You interact with it through the Docker CLI. That is great, but sometimes it can be nice to have a web interface to manage and visualize the cluster and containers. UCP is Docker's enterprise offering that is built on top of swarm mode to provide a web interface for cluster management and role-based access control. Because UCP is built on swarm, what you learn in this course applies to UCP as well.
All right, now we have a basic understanding of swarm mode. We will take closer look at how swarm mode works by understanding main components of its architecture in the next group of lessons.
Logan has been involved in software development and research since 2007 and has been in the cloud since 2012. He is an AWS Certified DevOps Engineer - Professional, AWS Certified Solutions Architect - Professional, Microsoft Certified Azure Solutions Architect Expert, MCSE: Cloud Platform and Infrastructure, Google Cloud Certified Associate Cloud Engineer, Certified Kubernetes Security Specialist (CKS), Certified Kubernetes Administrator (CKA), Certified Kubernetes Application Developer (CKAD), and Certified OpenStack Administrator (COA). He earned his Ph.D. studying design automation and enjoys all things tech.