image
Persistence

Contents

Intro
1
Introduction
PREVIEW3m 28s
2
Overview
PREVIEW3m 6s
Deploying Containers
Orchestration
7
8
Pods
3m 27s
10
Scenario Demo
13
Wrap Up
14
Summary
4m 28s
Start course
Difficulty
Beginner
Duration
1h 11m
Students
153
Ratings
4.4/5
starstarstarstarstar-half
Description

Container orchestration is a popular topic at the moment because containers can help to solve problems faced by development and operations teams. However, running containers in production at scale is a non-trivial task. Even with the introduction of orchestration tools, container management isn’t without challenges. Container orchestration is a newer concept for most companies, which means the learning curve is going to be steep. And while the learning curve may be steep, the effort should pay off in the form of standardized deployments, application isolation, and more.

This course is designed to make the learning curve a bit less steep. You'll learn how to use Marathon, a popular orchestration tool, to manage containers with DC/OS.

Learning Objectives

  • You should be able to deploy Mesos and Docker containers
  • You should understand how to use constraints
  • You should understand how to use health checks
  • You should be familiar with App groups and Pods
  • You should be able to perform a rolling upgrade
  • You should understand service discovery and load balancing

Intended Audience

  • Sysadmins
  • Developers
  • DevOps Engineers
  • Site Reliability Engineers

Prerequisites

To get the most from this course, you should already be familiar with DC/OS and containers and be comfortable with using the command line and with editing JSON.

Topics

Lecture What you'll learn
Intro What to expect from this course
Overview A review of container orchestration
Mesos Containers How to deploy Mesos containers
Docker Containers How to deploy Docker containers
Constraints How to constrain containers to certain agents
Health Checks How to ensure services are healthy
App Groups How to form app groups
Pods How to share networking and storage
Rolling Upgrades How to preform a rolling upgrade
Persistence How to use persistent storage
Service Discovery How to use service discovery
Load Balancing How to distribute traffic
Scenario Tie everything together
Summary How to keep learning

If you have thoughts or suggestions for this course, please contact Cloud Academy at support@cloudacademy.com.

Transcript

Welcome back. In this lesson, we'll look at persistence. If you're going to use stateful apps, then you need to think about persistent storage. DC/OS offers two options: persistent volumes and external volumes. Both options allow you to persist state, such that if a task fails, data will remain. External volumes are considered experimental in DC/OS 1.10. However, it's based on a project called REX-Ray, that allows you to use external storage providers, such as AWS EBS, GCE Disks, OpenStack Cinder, and others. Now, this is a great way to have data centrally located and persisted.

We're going to focus on local volumes for this lesson. Local volumes create a device on the agent that's running the application and shares it with a container. That means if a task fails, it needs to be restarted on the same agent. Also, these volumes aren't backed up by DC/OS. Actually, DC/OS doesn't manage them at all. That's on you, so if you want them deleted or backed up, you're going to have to take care of that.

So, for this demo, I want to show that if a task fails, it's going to restart on the same node, and it will have access to the same data as the original container. So, here's what I'm going to do. I'm going to add a gate to a file in a persistent volume, only if the file doesn't already exist.

So, if the file exists, it's not gonna edit the date. Then, I'll print the date in the file to standard out, so that you can see that the date doesn't change. And to simulate a failure, what I'll do is wait 30 seconds and then shut down the container.

Now, because I don't wanna do any of this manually, what I do is I have this one-liner here, and it's just gonna kind of wrap that functionality that I just mentioned. It's going to check to see if the /data/date file exists. If not, it will create it by sending the date into it. Then, it's going to cat the results to standard out. It'll sleep for 30 seconds, and then we'll shut down.

Other than knowing what that command is doing, it's not very interesting. What is interesting is the volumes property. This allows you to map a volume on the agent to a volume in the container. So, let's create this app. Okay, and let's head over to the UI. And great! It's already running. Let's look at the log, because standard out shows that it's going to have a date, and the date here is Saturday, September 23rd, at 20:23 UTC.

So now, in 30 seconds or so, this container's going to shut down, and even without a health check, that's considered a failure. So, I'm going to speed this up, just a bit, so you don't have to wait the full 30 seconds. And now, in just a second, the container's going to be killed and restarted. And there it was. Now it was quick, but the container is now back up and running. Let's look at the logs. And notice that standard out didn't change.

Now, because there wasn't a visual change in the log, it may seem like nothing actually happened. However, the container stopped, and since Marathon is supposed to keep containers in a running state, it started another one. Because there are persistent volumes, a failed task starts on the same agent. And so, it's started up another container, and then just rebounted that volume. And so, now you're seeing that the date is the same, because it was only set the first time.

On the Files Tab, you can see that there's a list of files created by the container. Here's our data folder, and the date file inside of that. And volumes are important for any sort of stateful application. They require some planning for backup, replication, and other management tasks. However, should you have some stateful workloads, you should start looking at volumes. Alright, in the next lesson, we're going to look at Service Discovery. So if you're ready to get started, I'll see you in the next lesson.

About the Author
Students
99369
Labs
32
Courses
45
Learning Paths
56

Ben Lambert is a software engineer and was previously the lead author for DevOps and Microsoft Azure training content at Cloud Academy. His courses and learning paths covered Cloud Ecosystem technologies such as DC/OS, configuration management tools, and containers. As a software engineer, Ben’s experience includes building highly available web and mobile apps. When he’s not building software, he’s hiking, camping, or creating video games.