In this lesson, I'll be talking about Modules. I'll cover what they are, and the different types. Modules are an individual unit of work, they represent things such as: creating files, creating virtual machines on a cloud platform, editing firewall rules, and just about anything else that you'd ever want to do.

Ansible has around 750 modules at the time of this recording. Everything that's done with Ansible happens with a module. When I showed how to set up the inventory file, I used the Ansible command to run the ping module. Because modules are basically a wrapper around executable code, most modules accept arguments in the form of key-value pairs.

Here are some examples of modules being run as ad-hoc commands. The first command targets the lamp groups, and it’s going to run the service module. There are two variables passed in, the name and the state. The name, in this case, is the apache2 service, and the state is started. If this was to run, it would make sure that the apache2 service is started.

The next module is ping, which doesn't have any variable, because we really don't need any. And the last one here is the command module, which allows you to run an executable on a remote machine. In this case, it’s using the reboot command. If you look at the documentation, you can see Ansible has modules broken down by category. These include modules for interacting with things like cloud platforms, files, storage, web infrastructure and Windows.

Now by drilling into any category, there's a list of individual modules. Looking at the file modules, you can see we have, for example, a module for setting access control, one for copying files, another for ensuring that a line of text exists inside of a file, and more.

Let's drill into a single module to better understand them. Let's use the copy module as our example. If I scroll down, there's a list of options that can be set with key-value variables. The table here has columns to indicate which are required and which ones are optional, and it shows the default values as well.

Scrolling down, we see that there are some examples of using the copy module in a play. That's something that will be covered later on. For now, it's just worth noting that modules tend to have plenty of examples of how to use them, so you're not going to be without documentation.

Ansible ships with a set of built-in modules, and it also comes with some extra modules that are community-supported and community-created. And if neither of these work out, you can also create your own custom modules. I mentioned before that Ansible ships with about 750 modules, however, if you need some functionality that isn't supported, you can easily create a custom module.

One of the things that I like best about Ansible is that they make it really easy to create custom modules. You can create modules using any language capable of writing JSON to STDOUT. That means you could use a shell script, PowerShell, or really just about any programming language.

When you tell Ansible to run a module against a host, it connects to that host, it copies the module code over, and then it executes that code on the host. So, if you're going to write modules in Ruby, for example, then Ruby needs to be installed on any hosts where that module will be run. Actually, let's go ahead and create module.

I'm going to create a few simple custom modules to show you how to get started developing them for yourself. When developing modules, it's best to use the test module script that Ansible provides, because it helps you to identify common mistakes such as writing non-JSON data to STDOUT, et cetera.

So, I need to download the Ansible source code to get that. For that, I'll use Git, and I'm going to download files from the Ansible repository. When downloading this, make sure you use the minus minus recursive flag to download any sub-modules. That will take a minute or two to download, so I'm going to fast-forward to when it's complete.

With the source code installed, I need to run a script to make sure the environment variables are configured for development. For that, I'll run source, and I'll pass it in the ENV set-up script in the Ansible/hacking directory. I'm going to create three modules. The first will be a module with a shell script, the second will be to use Python without the Ansible API, and then the final will be using Python with the Ansible API.

I'm using a bash script for this first demo, specifically because it's not a full-featured programming language. However, it does support file IO, and it can write JSON to STDOUT.

So, it meets Ansible's criteria for modules. The goal of this module is to count the number of files for a given file extension in the current directory. It's not a particularly useful module, however it does make for a reasonable demonstration. I've created this script offline so that you don't need to watch a bunch of boring typing.

When running modules, Ansible will pass the module a file name as an argument. The file it passes in stores any arguments for the module. The first section here checks if the arguments file exists, and then uses arg to parse the file and grab the value for the expected argument named ext and store it in the variable named file_ext. This module assumes the use of the key-value syntax for arguments.

The next section will check to see if the file_ext variable was populated in that previous section. If not, it's going to set the default to "py". Then, the file count variable will be populated by running a subshell to count files with the extension that was passed into the module. And the last line is going to print out a string formatted as JSON to STDOUT using echo.

Alright, let's test this. To test a module, you use the test module Python script, which behaves kind of like Ansible's ad-hoc command. The difference is that it will run the module locally, and it will print out any information to help you write better modules.

So, I'll run this by calling the test module and using the -m flag to specify the module, and then pass in the name of the module I want to test. I'm going to run this without any arguments to start off. When it runs, it returns the JSON object that was created on the last line of the script.

In this case, it shows a file count of 1,471 with the extension py. However, being able to pass arguments makes modules much more valuable, so I'll try it again and pass in the ext argument with a value of "yml", or yml. Just as you would with the Ansible command, you use the -a flag to pass in arguments.

Whatever argument string you pass in will end up in the arguments file. I'm using the key-value syntax where you specify the key name, an equal sign, and then the value. When this is run, it prints out the count of 533, and the extension of yml. And if I run it again with a different extension, it'll print out those results too.

Let's check out the next example. I'll demonstrate a version of the file count module I just showed, only in Python, without the Ansible API. This really could be any language, however I want to limit the number of languages used in this course. Just like the shell script in the previous example, this module is designed to accept a file extension to check for, however this module also accepts the directory to check in.

Let's walk through it. There are a few import statements at the top of the file, and then I've put the code into a main function. The function starts out the same way the shell script did, by storing the arguments file name in the variable. Then, it reads the contents of that file, and stores it in the argument content variable. Because Ansible just stores the argument in the file as a string, you need to choose how to parse those for yourself.

In this example I'm assuming these arguments will be passed in as key-value pairs, so I'm using the ShellEx library to parse the arguments into a list of strings. The ShellEx library is a simple lexical analysis library.

Next up, I'm looping over the arguments and breaking them up by key-value pairs, by splitting on the equal symbol. Those key-value pairs are being saved into the argument's dictionary. Automatically unpacking key-values by splitting on the equals symbol here is not a perfect solution, because if the value has an equals symbol in it, it's not going to be parsed correctly.

However, the goal of this isn't to demonstrate a perfectly thought-through module. Rather, it's to explain the types of things that you're going to need to handle when you don't use the Ansible API.

Now I have the arguments in a dictionary, so I can check to make sure that both the dir and ext arguments both exist. If either one doesn't, then I print a JSON object containing the error message to STDOUT, and then exit the script with a status of one. If the code makes it this far then both arguments exist, and so I need to run some code to get the count of files that have the extension matching what I've passed in.

Then, I wrap up the function by writing a JSON object to STDOUT, and it contains the directory, the extension, and the file count. With this created, I can test it out on the command line with the test module script. When I run this without any arguments, the module fails, and returns the error message because the module requires both dir and ext. If I run it again, and I provide a directory and the extension pattern, it runs again and returns the details.

So this is very similar to the bash script from earlier, except higher level languages tend to make a lot of tasks easier. When you create modules for yourself without the Ansible API, you're left solving things such as: argument parsing, writing JSON, making sure that only JSON is written to STDOUT, et cetera.

So you can write modules in whatever language you want, however it will mean you'll probably need to re-engineer the wheel at some point. Which is why I recommend that, if you can use Python, and the Ansible API, then you should do that.

I've refactored the module from the previous example offline, and I have them side-by-side for comparison. The version on the left is the previous example, and the one on the right is the example with the Ansible API. If you skim over the code on the right side, you'll notice that I'm not parsing the arguments file manually, and I don't need to explicitly write code to check for the existence of arguments. All of that is handled with the Ansible module class, this individual class has a lot of value.

Take a look at the argument spec dictionary. This is where you specify the parameters that your module is expecting. In this case, I'm expecting the directory and the extension, and they're both set to required. By setting required to true, Ansible will make sure that they're set for you, and then you can easily fetch the values from the params dictionary.

This new version also uses the exit_JSON method in the place of serializing to JSON and exiting the script manually. Ansible provides an exit_JSON method to easily write the output to JSON and end the script with a success status of zero. And if you need to fail a script, you could use the fail_JSON method and using these are considered best practice.

Let's go ahead and run this. If I execute this module without any arguments, Ansible fails because it's missing the arguments. And it's going to tell you which arguments are missing. If I run it with both arguments, then it completes as expected and returns the count of Python files on the current directory.

One of the cool things about Ansible, is that behind the scenes, it allows you to pass in arguments as key-value pairs or as JSON, and it doesn't require a code change. So, if I pass in the same arguments as JSON, this is still going to work. This allows the people using your modules to have the same flexibility they've come to expect from the core modules.

Now, when you're ready to actually use modules that you've created, you can copy them over to any directory mentioned in the ansible_library environment variable. Also, if you add a library directory in the same directory as your top-level playbook, Ansible will automatically look for modules there.

Hopefully this has shown you how easy it is the get started with custom modules. Treating modules kind of like Lego blocks to build complex playbooks is where the real value in Ansible is. And that's going to be the topic of our next lesson. So if you're ready to learn about playbooks, then let's get started in the next lesson.