The course is part of this learning path
DC/OS: Container Orchestration
Container orchestration is a popular topic at the moment because containers can help to solve problems faced by development and operations teams. However, running containers in production at scale is a non-trivial task. Even with the introduction of orchestration tools, container management isn’t without challenges. Container orchestration is a newer concept for most companies, which means the learning curve is going to be steep. And while the learning curve may be steep, the effort should pay off in the form of standardized deployments, application isolation, and more.
This course is designed to make the learning curve a bit less steep. You'll learn how to use Marathon, a popular orchestration tool, to manage containers with DC/OS.
- You should be able to deploy Mesos and Docker containers
- You should understand how to use constraints
- You should understand how to use health checks
- You should be familiar with App groups and Pods
- You should be able to perform a rolling upgrade
- You should understand service discovery and load balancing
- DevOps Engineers
- Site Reliability Engineers
- Familiar with DC/OS
- Familiar with containers
- Comfortable with the command line
- Comfortable with editing JSON
|Lecture||What you'll learn|
|Intro||What to expect from this course|
|Overview||A review of container orchestration|
|Mesos Containers||How to deploy Mesos containers|
|Docker Containers||How to deploy Docker containers|
|Constraints||How to constrain containers to certain agents|
|Health Checks||How to ensure services are healthy|
|App Groups||How to form app groups|
|Pods||How to share networking and storage|
|Rolling Upgrades||How to preform a rolling upgrade|
|Persistence||How to use persistent storage|
|Service Discovery||How to use service discovery|
|Load Balancing||How to distribute traffic|
|Scenario||Tie everything together|
|Summary||How to keep learning|
If you have thoughts or suggestions for this course, please contact Cloud Academy at email@example.com.
Welcome back. In this lesson we're going to test out Mesos containers from the command line and the UI. Mesos containers like Docker containers are based on C groups and namespaces. The difference is that Docker also offers a format for bundling up the application in a consistent way, so why would you ever select one over the other?
Now, there are some tasks where Docker adds overhead that isn't required and I'm not talking about resource overhead, rather overhead in that it introduces tooling that you may not need. Imagine you have a single binary that pings a service and then notifies you if it's not available. Now, while you could run that in a Docker container it's not required.
For tasks such as this Mesos containers work very well. Let's dive into actually creating and running a Mesos container. Starting here in the web UI, you can see that there are no services running. I wanna show you how to create an app with the CLI with the DCOS UI and then with the Marathon UI, so I'm going to create the same app three different ways.
Let's start with the CLI. To use the CLI you need to create a JSON file to represent the app. I have a JSON file already and if I display the contents here you can see that it's pretty basic. The ID is a unique identifier for the application and while you can't tell here from this example it supports namespaces.
So, if you wanted to break out an app you can use forward slashes. An example might be /apps/sales/funnel would be an app that is under the sales folder and that sales folder would be under the apps folder and this just helps you to organize your applications. A command property is the command that should be run in the container.
In this example the command is going to loop every five seconds and print out Hello Marathon followed by a date timestamp. It's not much, however, by looking at the logs we'll be able to see that the container's working. CPUs represent how many CPUs your application should allocate and this doesn't need to be a whole number.
As you can tell I'm only using a 10th of a CPU here in this example. Memory is in megabytes and here we're using 10 megabytes. Instances determines how many instances of this container will run and here I'm asking for two which means this command is going to be running twice. There are a lot more properties that you can use and you're going to see them throughout the course, however, I want to skim through them now so that you're at least familiar with them when you see them later.
First up is the container property which allows you to specify the type of container as well as information about the container. As an example, if you're using Docker you could set the image as well as portMappings. Next is the URIs. This property allows you to specify URIs in a list. Each URI will be downloaded onto the container and this is useful for a lot of different tasks though I find it works well with Mesos containers to download an application that you can extract and run with the CMD property.
An example of this would be if you had some operations tools that you want to run that aren't already wrapped in a Docker container. Now, I'm not going to read them off, however, you can see here that there are several protocols supported for URIs as well several archived formats that can optionally be automatically extracted.
The next property is the env property which as you may have guessed allows you to set environment variables. Next up is constraints which allow you to control which agents your containers will run on and this is a great feature and one we'll cover later in the course. Next is the Accepted ResourceRoles.
Roles are a Mesos concepts that represent a resource consumer. A role could represent an individual user, it could be a team, a department, it could also represent a service or some other entity. Then there are labels which allow you to set an arbitrary set of key value pairs that you can use to tag applications and finally, there is a healthCheck option that allows you to set up rules to determine whether apps are healthy or not.
Okay, so back to the task at hand. We're deploying a Mesos container. The command to deploy an app with Marathon is dcos marathon app add and then followed by the name of the JSON file that has the app definition. In this case the file's called basic. json. Okay, there we go, a deployment has been created, so to see the apps running with Marathon you can use the dcos marathon app list command.
Notice here that hello world is listed and there are two running instances. Jumping into the UI shows the same thing and by clicking on the service it shows the two tasks that were started. There are two because in the JSON file I set the instance count to two. By clicking on one of these you can see details about that task, however, the interesting part for us is on the logs tab.
Notice the output from standard out shows the message Hello Marathon followed by the date. This log updates without a page reload on some fixed timer and you can see that you can access standard error as well by clicking on the standard error filter. Heading back to the details page you can see information about the tasks such as the node that it's running on, where the endpoint is, any labels, Marathon information and more.
Now, speaking of Marathon, this brings us to the second way that we'll create a Mesos container. Marathon is a framework for managing long running tasks on Mesos and has its own UI that you can access at /marathon. Notice that it looks similar to the DCOS UI. You have a list of applications and there's currently just the one application with the two instances.
Let's use the Marathon UI to create another app called basic-from-marathon by clicking the create application button. So, here's the dialog, it requires a name which will be basic-from-marathon. I'll leave the CPU set to one, however, I want to change the memory. Do you recall how much memory I allocated for the Hello World app that we created in the command line?
If you don't, I had it set to 10 megabytes. Let's try and set that here to and no such luck. The Marathon UI enforces a requirement for at least 32 megs of RAM, so let's use 32. And I bring it up just so that you know there are some subtle differences. Let's set the instances to let's say three and for the command I have something copied in my clipboard here and I'll paste that and just change the text a bit.
Hello Marathon from the Marathon UI. There's nothing else required to set here. All of the defaults are going to work, however, you can see there are tabs for setting information about the Docker container, environment variables and things like that. On top of the dialog is a JSON mode option and that allows you to see and edit the JSON for this application.
Now, since all looks good let's click on create application. It's in the deployment stage now which will take just a moment and there it is, the app is running. Now, drilling into this you can see that there are three instances unlike the other app that only had two. In the DCOS UI there's an output log that updates automatically.
That's not the case here for the Marathon UI. You have to click on the link here and download the files as text files, so let's head back to the DCOS UI so that we can see the output without needing to download files. And there it is, there's our message Hello Marathon from the Marathon UI, so while we're here, let's go and create the same basic application with the DCOS UI so we can see the difference between the three.
So, from here on the services tab you can click the plus sign and it opens up this page where you can select the type of container to start. Let's ignore the other options for now and select the single container option. This next screen looks familiar because it's basically what we just did with the Marathon UI except it's a bit more condensed.
Let's start by naming the service /basic-marathon-dcos and there we go and let's see if the memory lets us set it as 10 megabytes and it seems to. So, the takeaway from that is that there are some differences in the constraints between the user interfaces that you should be aware of. The default here for CPU is also set to .
1 where it was set to one on the Marathon UI. Let's set the instance count here to five for this example. And I'll paste in this command and again, I'm just going to edit this, Hello from the DCOS UI. Okay, perfect. Clicking on more will reveal some additional container options. Notice that the runtime is set to Docker and that's not what we need, so there's also a Mesos container option and that's the one we're going to want which again, is based on control groups and namespaces.
So, skimming through the other tabs you can see there's nothing else we need for this app. Just like the Marathon UI, there's a JSON editor should you want to copy or edit the JSON. Okay, let's create this application by clicking review and clicking run. Okay, notice it's already up and running. Clicking into the service you can see that there are five instances and drilling into any of these will show the same output message.
Alright, now we have all of these up and running. We've created them three different ways. Let's clean up after ourselves by deleting these services. Since I showed you how to create these in three different ways it makes sense that we should remove them all in the same way they were created. So, starting here on this screen, clicking on the ellipsis icon has an option to destroy the service.
It prompts for confirmation and there we go, now let's just click on this scary red button to remove the service and it's already gone. So, let's delete the next service from the Marathon UI. And here we go, so again we click on the ellipsis icon and select destroy and once again there's a terrifying red button, so let's click that and away goes the service.
Okay, let's delete the original app on the command line and for that you can use the dcos marathon app remove command followed by the ID of the service you want to remove and there we go. Now, to prove that it's gone, you can run the dcos marathon app list command which is blank as expected. Okay, let's wrap up here.
In this lesson you saw how to create and destroy Mesos containers using the command line interface, the DCOS user interface and the Marathon user interface. In the next lesson we're going to switch from Mesos containers to Docker containers, so if you're ready to run some Docker containers on DCOS, then I'll see you in the next lesson.
About the Author
Ben Lambert is the Director of Engineering and was previously the lead author for DevOps and Microsoft Azure training content at Cloud Academy. His courses and learning paths covered Cloud Ecosystem technologies such as DC/OS, configuration management tools, and containers. As a software engineer, Ben’s experience includes building highly available web and mobile apps.
When he’s not building the first platform to run and measure enterprise transformation initiatives at Cloud Academy, he’s hiking, camping, or creating video games.