Container orchestration is a popular topic at the moment because containers can help to solve problems faced by development and operations teams. However, running containers in production at scale is a non-trivial task. Even with the introduction of orchestration tools, container management isn’t without challenges. Container orchestration is a newer concept for most companies, which means the learning curve is going to be steep. And while the learning curve may be steep, the effort should pay off in the form of standardized deployments, application isolation, and more.
This course is designed to make the learning curve a bit less steep. You'll learn how to use Marathon, a popular orchestration tool, to manage containers with DC/OS.
Learning Objectives
- You should be able to deploy Mesos and Docker containers
- You should understand how to use constraints
- You should understand how to use health checks
- You should be familiar with App groups and Pods
- You should be able to perform a rolling upgrade
- You should understand service discovery and load balancing
Intended Audience
- Sysadmins
- Developers
- DevOps Engineers
- Site Reliability Engineers
Prerequisites
To get the most from this course, you should already be familiar with DC/OS and containers and be comfortable with using the command line and with editing JSON.
Topics
Lecture | What you'll learn |
---|---|
Intro | What to expect from this course |
Overview | A review of container orchestration |
Mesos Containers | How to deploy Mesos containers |
Docker Containers | How to deploy Docker containers |
Constraints | How to constrain containers to certain agents |
Health Checks | How to ensure services are healthy |
App Groups | How to form app groups |
Pods | How to share networking and storage |
Rolling Upgrades | How to preform a rolling upgrade |
Persistence | How to use persistent storage |
Service Discovery | How to use service discovery |
Load Balancing | How to distribute traffic |
Scenario | Tie everything together |
Summary | How to keep learning |
If you have thoughts or suggestions for this course, please contact Cloud Academy at support@cloudacademy.com.
Welcome back. In this lesson, we'll look at persistence. If you're going to use stateful apps, then you need to think about persistent storage. DC/OS offers two options: persistent volumes and external volumes. Both options allow you to persist state, such that if a task fails, data will remain. External volumes are considered experimental in DC/OS 1.10. However, it's based on a project called REX-Ray, that allows you to use external storage providers, such as AWS EBS, GCE Disks, OpenStack Cinder, and others. Now, this is a great way to have data centrally located and persisted.
We're going to focus on local volumes for this lesson. Local volumes create a device on the agent that's running the application and shares it with a container. That means if a task fails, it needs to be restarted on the same agent. Also, these volumes aren't backed up by DC/OS. Actually, DC/OS doesn't manage them at all. That's on you, so if you want them deleted or backed up, you're going to have to take care of that.
So, for this demo, I want to show that if a task fails, it's going to restart on the same node, and it will have access to the same data as the original container. So, here's what I'm going to do. I'm going to add a gate to a file in a persistent volume, only if the file doesn't already exist.
So, if the file exists, it's not gonna edit the date. Then, I'll print the date in the file to standard out, so that you can see that the date doesn't change. And to simulate a failure, what I'll do is wait 30 seconds and then shut down the container.
Now, because I don't wanna do any of this manually, what I do is I have this one-liner here, and it's just gonna kind of wrap that functionality that I just mentioned. It's going to check to see if the /data/date file exists. If not, it will create it by sending the date into it. Then, it's going to cat the results to standard out. It'll sleep for 30 seconds, and then we'll shut down.
Other than knowing what that command is doing, it's not very interesting. What is interesting is the volumes property. This allows you to map a volume on the agent to a volume in the container. So, let's create this app. Okay, and let's head over to the UI. And great! It's already running. Let's look at the log, because standard out shows that it's going to have a date, and the date here is Saturday, September 23rd, at 20:23 UTC.
So now, in 30 seconds or so, this container's going to shut down, and even without a health check, that's considered a failure. So, I'm going to speed this up, just a bit, so you don't have to wait the full 30 seconds. And now, in just a second, the container's going to be killed and restarted. And there it was. Now it was quick, but the container is now back up and running. Let's look at the logs. And notice that standard out didn't change.
Now, because there wasn't a visual change in the log, it may seem like nothing actually happened. However, the container stopped, and since Marathon is supposed to keep containers in a running state, it started another one. Because there are persistent volumes, a failed task starts on the same agent. And so, it's started up another container, and then just rebounted that volume. And so, now you're seeing that the date is the same, because it was only set the first time.
On the Files Tab, you can see that there's a list of files created by the container. Here's our data folder, and the date file inside of that. And volumes are important for any sort of stateful application. They require some planning for backup, replication, and other management tasks. However, should you have some stateful workloads, you should start looking at volumes. Alright, in the next lesson, we're going to look at Service Discovery. So if you're ready to get started, I'll see you in the next lesson.
Ben Lambert is a software engineer and was previously the lead author for DevOps and Microsoft Azure training content at Cloud Academy. His courses and learning paths covered Cloud Ecosystem technologies such as DC/OS, configuration management tools, and containers. As a software engineer, Ben’s experience includes building highly available web and mobile apps. When he’s not building software, he’s hiking, camping, or creating video games.