Linux Shell Scripting
The course is part of this learning path
In this course, you're going to learn how to set up a local Linux shell scripting environment, whether you're using Windows, MacOS, or Red Hat Enterprise Linux. You'll be guided through the installation processes of Vagrant and VirtualBox—two tools that we will make your life easier as we work our way through the Linux Shell Scripting learning path. We'll also carry out some troubleshooting of the most common issues, should you face them.
This course is part of the Linux Shell Scripting learning path. To follow along with this course, you can download all the necessary resources here.
- Set up a local Linux environment using Vagrant and VirtualBox
- Troubleshoot any issues/errors
- Anyone who wants to learn Linux shell scripting
- Linux system administrators, developers, or programmers
To get the most out of this course, you should have a basic understanding of the Linux command line.
During this course, you're going to write several shell scripts and learn a lot along the way. It's really important that you learn the concepts so that you can apply your knowledge on any Linux distribution or any Unix slide or Linux like system. However, when you're learning a new concept, I don't want anything to get in your way. I don't want you to get hung up on little gotchas or get sidetracked by some unimportant detail. That's the main reason why you're going to use a specific Linux installation that was created for this course. Another reason you're going to use the Linux image designed for this course is so that you can get help. If everyone in the class were to use a slightly different Linux installation, it would be almost impossible to manage the class. If I are the teaching assistants know exactly what system you're using, we can help you in a very specific way. We can even take your script and run it on a system that is the same as the one you're working on. This helps us help you. A third reason to use the Linux installation method I'm about to describe is that it will allow you to quickly create an entire mini network of Linux systems. In the real world ,you'll end up writing shell scripts that perform a given task on multiple systems. This setup allows you to practice those skills and we're gonna do that in this class. Okay, so what does this installation method and lab set up that I've been alluding to for the last couple of minutes here and what this is environment look like? First, it consists of virtualization software that will allow you to run Linux inside a virtual machine on your local desktop or laptop. My favorite virtualization software for this purpose is VirtualBox available for firstname.lastname@example.org. First off, it's free to download and it's really easy to install. Secondly, it runs on windows, Mac and Linux. This is a big plus for a course like this as you're going to have the exact same experience, no matter if you're working on a windows system, a Mac system, or if you use Linux on your desktop every day. A little sign out here, if you're looking for a virtualization solution for running virtual machines in production for example, I'd be looking at something like VMware, KVM or oVirt, but for our purposes for creating test systems on our local machine and working together in this class, VirtualBox is the way to go. Let's get clear on what VirtualBox does and what virtual machines are. VirtualBox is a full virtualization product, meaning that it allows you to run an unmodified operating system with all of its installed software in a special environment on top of your existing operating system. This special environment called a virtual machine, is created by the virtualization software by intercepting access to certain hardware components and certain features. The physical computer is called the host, while the virtual machine is called a guest. Most of the guests code runs on modified directly on the host computer, and the guest operating system thinks it's running on a real machine. That last point is important. The guest operating system thinks it's running on real hardware, and it really doesn't know that it's inside a virtual machine. If you're doing something inside a virtual machine and the operating system of that virtual machine says something like, performing this action will erase all the data on all your disks. It's really only referring to the virtual disks that it has access to. Not your physical disk on your real physical host computer. You can perform destructive actions inside the virtual machine, and those actions are contained inside of that virtual machine and won't interfere with your actual physical computer. VirtualBox provide software that can be installed on the guest operating system. This software is called VirtualBox guest additions. This software allows the guest operating system to access shared folders on the host system, share the clipboard and some other similar actions. This is not required, but it can be very useful. The virtual machines you'll be deploying in this course, we'll already have the guest edition software installed in them. I just wanted to mention the guest addition so that you're aware of them and know what they do. Manually creating virtual machines with VirtualBox is fairly easy and straightforward. By the way, we are not going to be manually creating machines in this course. I just wanna show you the process so that you can do this on your own later or so that you know exactly what is happening behind the scenes when we automate this. So first you start the VirtualBox application and next you click on new to create a new virtual machine. From there you would give it a name, tell VirtualBox a bit about the operating system that you'll be installing inside of the virtual machine and allocate some resources such as memory and disk based to the virtual machine. When you create a virtual disk for the virtual machine, you can make it dynamically allocated. A virtual disk is simply a file on the host system. When you choose to dynamically allocate that disk, it will only use space on your physical disk as that space is used inside the virtual machine guest. This means that the disk file will grow as needed. The upside is that the virtual machine thinks it's getting a large disk, such as a hundred gigabyte or 200 gigabyte. But if it only uses one gigabyte, then only one gigabyte of space, is used on your physical hard drive. The price you pay for this dynamic allocation is performance inside the virtual machine, but for our purposes this is totally acceptable. Once you've allocated resources to the virtual machine, it's ready to be used. And the first time you start it up, it will ask you to attach a disk to boot from. Typically what you'll do is download a Linux ISO from your favorite distributions website, and then tell VirtualBox to use that ISO to boot the system with. Alternatively, you can actually put a physical CD or DVD in your computer and tell a VirtualBox to use that physical drive for the install. Now you install the OS and the virtual machine just like you would as if you were installing that OS on real physical hardware. This process is all well and good but as you can probably guess, it can be a bit time consuming going through the virtual machine creation process and manually performing the operating system installation each time you wanna create a new virtual machine. We're looking to automate this process and save us a lot of time doing that tedious work. To do that, we'll use a command line tool called Vagrant which is available to download for free from vagrantup.com. This command line tool works on Mac windows and Linux. So, no matter what OS you're using for this course, you'll be covered. Vagrant provides easy to configure, reproducible and portable work environments. By default Vagrant provisions of virtual machines using VirtualBox, it can be configured to deploy virtual machines to VMware, AWS, Rackspace and other providers. But for this course, we're sticking to creating virtual machines inside a VirtualBox running on our local systems. You're going to use Vagrant to quickly and easily spin up virtual machines running Linux. Let's talk about what you should know and understand about Vagrant before you start using it. A box in Vagrant speak is an operating system image. The Vagrant box add command will download and store that box on your local system. You only need to download a box once as this image will be cloned when you create a new virtual machine with Vagrant using the boxes name. The typical format of this command is Vagrant box add user ford/box. I have a user account on vagrants public box catalog system. My username is Jason C. I created a box specifically for this course named Centos seven. It's really just a minimal Centos installed. So to download that box, would run Vagrant box add Jason C, ford/centos seven. There are several boxes that are publicly available on vigorous public box catalog system. I'll have a link in the exercise instructions where you can download other boxes if you'd like to experiment with them on your own time. In addition to boxes, Vagrant uses the concept of projects. A Vagrant project must consist of a folder and a Vagrant configuration file called a Vagrant file. So to create a new Vagrant project, first create a folder. For example, I would open up a command prompt on windows or a terminal on Linux and Mac and then type the command, MKDIR space VM1. From there, you would change into that directory with CDVM1. Now you need to initialize your Vagrant project, which means creating a Vagrant configuration file. This configuration file is named Vagrant file, and it contains the box name to use for the project and any other configuration options for the virtual machine or virtual machines to create. To create the Vagrant file run the Vagrant init command. So, continuing with our previous example here, you would run Vagrant init, Jason C, ford/centos seven. Now you can bring up an instance of that box by running Vagrant up. The first time you run the Vagrant up command, Vagrant will import or clone that Vagrant box into VirtualBox and then start it. If Vagrant detects that the virtual machine already exists in VirtualBox, it will just simply start it. By default when the virtual machine has started, it has started in headless mode, meaning that there is no user interface for the machine visible on your local host machine. Since this class is focused on shell scripting, we'll be working at the command line. So, we really don't care about looking at the Linux desktop anyway. Now if you look at the VirtualBox application, you'll see that you'll have a new virtual machine that's running. It's name starts with the name of the Vagrant project which is the name of the director you created where the Vagrant file live. As I mentioned earlier, you are going to be creating multiple virtual machines. You can actually specify those additional virtual machines in the same Vagrant file. This is called a multi machine setup by Vagrant. By default, when you run the Vagrant up command, it brings up all the virtual machines defined in the Vagrant file. If you only want to operate on a single virtual machine, you have to supply the virtual machines name. For example, if you defined a VM named master, you could bring it up with a Vagrant up master command. Likewise, if you had a VM defined with the name of server one, you would bring it up with Vagrant up server one. And I wanna repeat that if you just ran Vagrant up, it would bring up both of those systems that were defined being caused by default, it brings up every system that's defined in the Vagrant file for that Vagrant project. SSH secure shell, is the network protocol used to connect a Linux systems. Vagrant provides a nice shortcut to SSH into the virtual machines. If you only have one virtual machine defined in the Vagrant file, you can simply run Vagrant SSH and admit the VM name and you'll be automatically logged into that one VM Linux system that you've created with Vagrant. If you have multiple machines defined, you'll need to supply the specific VM name, for example, Vagrant SSH master. To log out of a Linux system simply type exit and hit enter. A quick note for you. Windows users, windows doesn't currently ship with an SSH client. So you'll need to install one. And what I recommend doing is installing git, that's GIT, from git-scm.com, which includes an SSH client. When you work on the exercise, I'll give you specific instructions on how to get this configured. But for you Mac and Linux users, you can just skip this step as you already have an SSH client built in by default. Okay, back to Vagrant. Vagrant can be used to control the virtual machines that creates, for example, the Vagrant halt to command, shuts down all the virtual machines defined in the Vagrant file. If you have multiple VMs defined and only want to halt one, just specify the VM name. When you run this command, you will not lose any work you've performed inside the virtual machine. That's because the virtual machine will still exist in VirtualBox, but it will just simply be halted or stopped. To restart all the defined VMs for the project, run Vagrant up. Since the virtual machines already exist, a Vagrant up simply start them, and it will skip the import process because it's not needed. By the way, I'm pretty sure you see a pattern here with all this all command. If you don't specify a VM name, the command will operate on all the VMs defined in the Vagrant file for the project. If you wanna limit that operation to a specific VM, then you need to supply that name of the VM that you wanna operate on. So going forward, I'm just going to assume that you understand this concept and not pointed out for each and every command. If you want to suspend the virtual machine, run Vagrant suspend, to resume it, run Vagrant resume. If you're done with the virtual machine, or you want to start over with a fresh copy of the virtual machine of virtual machines, run Vagrant destroy. This will terminate and remove the virtual machine or virtual machines from VirtualBox. If you run this command, then any work inside of the virtual machine, will be lost. If you need help or forget some of the commands, you can just run the Vagrant command itself without any options. Also keep in mind that the vigorous command will act on the virtual machine and virtual machines associated with a Vagrant project directory that you're in. During this course, you'll be creating multiple Vagrant projects. So, be sure that you're in the desired Vagrant project directory before running any Vagrant commands. Before we wrap it up talking about Vagrant, let's take a quick look at the Vagrant file. All the vagrant configuration occurs after the vagrant.configureline, and before the end keyword. The two in parentheses here, is the configuration version. Older style configurations are supported but we'll be using version two. So this is what you'll see in the Vagrant file you'll be working with. The simplest Vagrant file just specifies the name of the box you'll use to create the default VM. If you want Vagrant to set the host name inside the virtual machine, you can use config.vm.hostname. To assign a static IP address to the virtual machine, there's a config.vm.networkline. The example on your screen, we'll assign the IP address of 10.2.3.4, to the virtual machine on a private network that can only be accessed from the host machine or other Vagrant VMs on that same network. You'll be using this Vagrant feature to create an entire network of Linux systems that can communicate with each other. Each box ships with a set of defaults, like whether or not to start in headless mode or how much memory to allocate to the virtual machine and so on. You can override these default settings by using a config.vm.provider block. The example here on your screen will force the virtual machine to start with a GUI, a graphical user interface, and use 1,024 megabytes, which is of course one gigabyte of memory. Vagrant has a provisioning system built in it. Provisioners and Vagrant allow you to automatically install software, alta configurations and more on the machine as part of the Vagrant up process. Because we're focusing on shell scripting in this course, we'll use the shell provisioner. You won't be using a provisioner all the time, but we will do this at some point during the course, this is where you can write a post insulation shell script to configure a new system for example. Vagrant will execute the shell script that you specify the first time Vagrant up is executed. The example here on your screen, we'll upload the setup.sh, shell script, into the virtual machine, and then execute it from there. This way you can automate and reproduce entire environments quickly. If you wanna make sure someone else like a developer is working with the same configuration as you, give them the Vagrant file and the setup.sh script and they're set. Let's look at how to configure a multi machine setup in Vagrant. If there is any common configuration to all the VMs, just specify it at the top of the file. In this example, we're going to use the same box or image for all the VM so we list it first. Next create a configuration block for the first VM. The syntax is shown on your screen. That syntax is config.vm.define. And then you put a server name in quotation marks followed by do, and then in a set of pipes, you also put the server name. Then you set the VM specific configuration using that server name. This example sets the host name and IP address for the VM. You keep adding additional VMs using the exact same syntax. On your screen, a second VM is defined named server two. If you were to run Vagrant up using this configuration, you would end up with two virtual machines. One named server one with an IP address of 10.2.3.4, and another one named server two with an IP address of 10.2.3.5. We've talked about many of the awesome things Vagrant can do and many of the Vagrant features you'll be using most often. However, there is more. The vagrant init command creates a default Vagrant file which contains several comments. You can read through those comments to get an idea of what else is possible with Vagrant.
Jason is the founder of the Linux Training Academy as well as the author of "Linux for Beginners" and "Command Line Kung Fu." He has over 20 years of professional Linux experience, having worked for industry leaders such as Hewlett-Packard, Xerox, UPS, FireEye, and Amazon.com. Nothing gives him more satisfaction than knowing he has helped thousands of IT professionals level up their careers through his many books and courses.