Linux Shell Scripting
The course is part of this learning path
In this course, you'll learn how to generate some random data, including how to automate the process of generating a random password. We'll then look at a range of shell script statements and parameters including positional parameters, arguments, for loops, special parameters, while loops, infinite loops, shifting, and sleeping. Finally, we'll walk you through how to add users to a Linux system.
This course is part of the Linux Shell Scripting learning path. To follow along with this course, you can download all the necessary resources here.
- Automate the process of generating a random password
- Learn about a variety of statements and parameters that can be used in shell scripting
- Learn how to add a user to a Linux system
- Anyone who wants to learn Linux shell scripting
- Linux system administrators, developers, or programmers
To get the most out of this course, you should have a basic understanding of the Linux command line.
In this lesson, you're going to be introduced to positional parameters and the for-loop. Here on my local system, I have a terminal open and I'm going to move into our class working folder. And from here into our Vagrant project folder, this project being local users. And then now I'm going to start the virtual machine and then connect to it. Now we'll move into the shared folder of cd/Vagrant. And this script, I'm going to name luser-demo06.sh. Of course, all scripts, we start with a shebang. So we'll give the path to the Bash interpreter. And then I'm just going to list the goal of this script or what this script does at the top here as our header. And the first thing we're going to do here is simply display what the user typed on the command line. And we can do that with the Echo command here. The variable of dollar zero is actually a positional parameter and positional parameters are variables that contain the contents of the command line. Before we go any further, let's talk about the definition of a parameter versus an argument. A parameter is a variable that is being used inside the shell script. And argument is the data passed into the shell script. So an argument supplied on the command line becomes the value stored in a parameter. With that in mind the very first positional parameter is dollar zero which contains the name of the script itself. You can think of this as being what the person typed in the command line to execute the script. The next positional parameter is dollar sign one which stores the value of the first argument passed to the script on the command line. The positional parameter dollar sign two stores the second argument, dollar sign three store the third argument and so on. So now let's look at the output of our script so far. Of course, we have to add executable permissions before we can execute our scripts. So we'll do chmod, and then execute the script. So notice how dollar sign zero expanded to how we executed the script, this dot forward slash and then the script name. Now let's change how we call the script by providing the full path to the script. So we can type /Vagrant luser-demo06.sh and hit Enter. So again, when the script is called in an explicit way like this, then that is what is stored in dollar sign zero. So dollar sign zero appears exactly as we have typed on the command line here. We're about to take a slight detour for a moment. Bear with me here because this will all tie into positional parameters. But again, it's going to take several minutes to explain this. So again, stick with me and be patient with me. So one thing I've glossed over is the command search path which is stored in the environment variable path. That's PATH in all uppercase letters. Up until this point, we've just been supplying either a full or relative path to the scripts we've created. Let's take a quick look at what the Bash Man Page has to say about the path variable before we move forward. Okay, we'll do a forward search with a forward slash followed by path. We'll just keep hitting in until we get to the path variable, and here it is. It says that path path is a colon separated list of directories in which the shell looks for commands. Now, if you were to look at the command execution section that it references here on your screen, you'll find that every time you type something in at the command line, Bash first tries to find a function with that name and execute it. Now we haven't covered functions yet but don't worry, we will. So if it doesn't find a function with that name, it looks for that command and its list of built-in commands. If it is a shell built-in command, then it executes that. Now you've been working with several built-in commands such as Echo, Read, Type, Help, and so on. Now, if the command you typed at the command prompt or on the command line is not a function or it's not a built-in, then Bash searches through the list of directories defined in the path variable and executes the first match that it finds. If no match is found, then a command Not Found error message is displayed. So let's get out of the Man Page here by typing Q and let's look at our path, echo dollar path. So in our particular case here, if a command is not a function or shell built-in, then Bash will look for that command first in user local bin, then in user bin, then it will look and use your local sbin, and so on down the list. It's important to point out that the path can be changed. So it's not something that is guaranteed to be the same for every user across every system. So keep that in mind. By the way, if you want to control the path, you can do so in your script by manipulating the path variable there. Let's use a neat, little command called Which to tell us which command will be executed. Let's take the Head command that we've used before as an example, which head. So this time tells us that when we type in head at the command line, the program located at user bin head will be executed. The first directory in our path variable is actually user local bin. So let's create our own shell script named head and place it in there. We'll need root privileges to do that. So I'm going to create the file with sudo, vim, user, local, bin, head. And then here we're just going to create a little shell script, hello from my head. And then we'll go ahead and save it. We'll add the executable bit to it here. Now let's see which head will be executed. So we'll type in which head, and it says user local bin head will be executed. Let's use the dash a option to Which which forces it to print all matching executables that are in our path. So there are the two commands named the exact same thing, both are named Head, but the first one that is found is executed. So let's go ahead and just execute this command. Type in head and hit Enter. Sure enough, our version of the head command is executed. Now, if you want to overwrite the default, then just supply the full path. So we can do that with user bin head. Let's say, we just want the first line of the Etsy password file, so we'll execute that. So this is what we've been doing all along which is explicitly defining a path to a command or actually to our scripts. Let's get rid of the version of our head. So we'll just use sudo rm user local bin head. Here is one little quirk with rapidly adding and removing programs from your path. Bash uses a hash table to remember the full path names of executable files. It looks up to the location to the executable once and then remembers it. If you look at the Type Head command which we'll do here, Type Head, it says that head is hashed to user local bin head. That simply means that Bash will execute that command at the path without first searching for that command. So here's the quirk, Bash thinks that head is still located at user local bin head. However, we just deleted it just a few seconds ago. So let's execute the command and see what happens. So if we type in head and hit Enter, it says user local bin had no such file or directory. So if you run into this type of issue, you can make Bash forget all these hashed locations by using the hash built-in with hash -r. Now, if we do type head, we can see that it searches the path and it finds the one that really is there in user bin. Of course, you can read about the hash shell built-in with the command Help Hash. And there you can see the -r option that we use so as to forget all remembered locations. Now typically, hashing is not a problem because commands don't usually just randomly appear and disappear during the same shell session and probably not during the execution of a shell script. In any case, it's just something to be aware of. Okay, so I know that was a rather long side but let's bring this back around to positional parameters and specifically about dollar sign zero. First, let's prove that our script is not in our current path. We can do that with a Which command, which luser-demo06, and sure enough, there is no luser-demo06 in our path and it lists our path there, user local bin, user bin, and so on. Now let's take a copy of our script and place it into our path by putting it into user local bin. And we can do that with sudo cp for copy, and then provide the path to the directory we want to copy it to, and hit Enter. Now our script is in our path so we can execute it without supplying a path to the script. Now notice what's stored in dollar sign zero. It's the full path to the command which is user local bin luser-demo06.sh. Even though we only typed in luser-demo06.sh at the command line, Bash found it in our path and executed at that path. And that's why the path in the file is stored in dollar sign zero. So let's review what gets stored in dollar sign zero in different situations. So we'll do ./luser-demo06. So dot forward slash is what is executed and that's what's in dollar sign zero. If we're explicit again and provide a full path such as the /Vagrant luser-demo06, that will be reported as well. However, if a command is our path and we execute it like so, then it reports the full path to that command. Okay, before we move on, let's remove the script out of our path before we continue. So we'll do sudo rm user local bin luser-demo06.sh. There's a command called Basename that removes the directory component and just returns the file name. So let's read up about that command, Man Basename. Here at the top of your screen, you can see that Basename strips out the directory and suffix from file names. So let's see what happens when we execute this. We'll just hit Q to get all the help page there, and we'll type Basename Vagrant luser-demo06.sh, and then it just returns just the file name without the directory portion of the file. By the way, Basename isn't doing any smart checking to see if the file exists, it just strips off the directory portion of a string that is passed to it. So we can give a Basename something that clearly doesn't exist, but it will still do its job. So we can do basename /not/here. And of course it reports here because that's the file name portion of that path to that file, even though the file doesn't exist. And we can prove that with ls not here. Sure enough, there's no such file or directory. The counterpart to the Basename command is DIRname. So let's look at the Man Page for that. This says that DIRname strips the last component from the file name. Let's hit Q to exit out of the Man Page and execute the commands. So we'll type DIRname Vagrant luser-demo06 and hit Enter. And forward slash Vagrant is returned because that is the directory name or the directory portion of that full path. Like Basename, DIRname doesn't do any smart checking. We can give it a non-existent path and it will simply return the portion that looks like the directory. So we can do this, DIRname, this is not here. Sure enough, it returns to this is not because it looks like the here file is in the directory /this/is/not. But of course, that file and directory, that whole tree doesn't exist, but DIRname doesn't check, it doesn't care. It just returns the path or the directory of the file. Again, obviously this works on things that do exist but it doesn't matter to the DIRname command or Basename command. Now let's use these two commands in our script. So what we're going to do is display the path and file name of the script. In earlier lessons, you use the dollar sign followed by a command that was wrapped in parentheses to assign the output of that command to a variable. That is called command substitution. In this case, we're only going to use the output of each of these commands once, so we just included them right in the Echo command. Again, what this dollar sign parentheses syntax does or this command substitution syntax does is return the output of the command. You can assign the output to a variable or use that output without assigning it to a variable. Either way is fine. Perhaps assigning the output of the commands to a variable first makes for a bit cleaner code. But I just wanted to show you that you don't have to use command substitution in variable assignment only, you can use it in line as well. So let's save and execute our script and see what we get here. The first line says you executed this command which is dollar sign zero .luser-demo06.sh. You used dot as the path to the luser-demo06.sh script. So dot was the output of the DIRname command and luser-demo06.sh was the output of the Basename command. So let's try executing this script in another way. We'll provide the full path /Vagrant luser-demo06.sh. So this time it lists /Vagrant as the path, and then of course the Basename of the script is luser-demo06.sh, okay? Let's look at the Bash Man Page real quick. And we're looking for the special parameters section of the page here, and then we'll keep hitting N to continue to our next match. And then the thing I want to point out here is the pound sign. And the pound sign, it says expands to the number of positional parameters in decimal. So dollar sign pound sign tells us how many arguments were supplied on the command line. Let's continue with our script here. I'm gonna use bang V to execute the most recent command that starts with a V which is Vim, so I can get back into my editor really quickly here. So let's tell the user how many arguments they passed in. Remember that curly braces are optional. So some people might actually just use dollar sign hash sign, so just be aware of that if you're reading other people's scripts, but I'm willing to use the curly braces here to be very clear. Of course, here you could have simply used the dollar sign hash sign directly in the Echo command, similar to what we did with a command substitution and the previous Echo command. It's really a style choice. And I just wanted do you see how you can do it both ways. So the first time I didn't use a variable, this time I used a variable and then use the value of that variable in the Echo command. So let's save our changes and then do some testing here. So we'll just execute our script. It says you supplied zero arguments on the command line. So let's just type something in after the command line, we'll just type hello. And it says, we supplied one argument on the command line and we'll type in another word, you supplied two arguments and so on. So this one says you supplied three and that is giving us the proper information. So it looks like dollar sign hash sign works and we're all good there. So let's continue editing our screen. Because the goal of this little example shell script that we're working on is to generate a random password for each user that's supplied on the command line, we want to make sure that at least one user is supplied on the command line. So let's do a check with an If statement. So use F space, double opening brackets, space. When we use a variable in between brackets here, we're going to enclose it in quotation marks. And the variable we're going to look at is number of parameters. And then we want to make sure that that's -lt which stands for at less than one. So if the number of parameters is less than one, then what we're going to do is echo a usage statement much like you would see in a Man Page. Instead of manually typing in the name of the script, we can just simply use dollar sign zero to echo back more or less what the user typed in at the command line. And then we'll tell them what they need to do. They need to supply a username and optionally, they can supply another username and actually optionally, they can supply even more usernames. So we'll use the ellipsis there. Now, if they get to this point, that means that the script did not successfully execute because they didn't give us any people to create passwords for. So what we're going to do is exit with a non-zero exit status, and I'm just going to use one here. Close our If statement with fi. Now, by the way, let's look at the first line here again. If the number of parameters is less than one, then blah, blah blah. So here, what I'm actually doing is just demonstrating something that we haven't used before, just a slight variation of -lt. And where did I get that from? Well, hopefully you remember that you can get help on these tests by looking at Help Test at the command line. Of course, you'll find in there that -lt stands for less than. Of course, what is the only thing that's less than one in our particular situation working with whole numbers here on the command line, negative numbers don't make sense because you can't supply negative arguments in this sort of thing. So obviously, the only thing that's less than one is zero. So you could have said if the number of parameters equals zero, then give them the usage statement and exit. So again, either way is right. It's just your personal preference. How you think, what makes sense to you, its style, again, both ways work, just whatever comes to mind, just use that. Again, I'm using this here, so you can see some slight variations and different ways to do this. Now let's save our changes and start passing in varying number of arguments to our script. We'll hit Enter, we did not use any arguments on the command line and we get our usage statement. And if we check the return code with dollar sign question mark or you can actually do this, dollar sign question mark with curly braces, then sure enough we get an exit status of one from our script since it didn't successfully execute all the way. So let's get back to our script and let's just give it one thing. Sure enough, you supplied one argument to, let's do A, B, C, D. Okay, that's for arguments. And I want to point out something here that we used actually with a User Add command in an earlier script which has enclosing things in quotation marks. So A, B, C, D. So says that you supplied three arguments on the command line, but maybe at first glance, you think that B and C are two different things, but in reality because they're enclosed in quotation marks, they're treated as one argument. Let's do this again. Let's do this one. This whole thing is two, and here is three. So again, in this instance, we supplied three arguments on the command line. Another thing you can do is simply enclose everything in quotation marks, and that's fine as well. I'm just going to use single quotation marks 'cause that works too. We'll do one, this is two and this is three. So again, three arguments on the command line which interprets to three positional parameters inside the script. There are three because they are enclosed in quotes. So it doesn't split it spaces when the data passed in is enclosed in quotation marks. Now, again, the idea behind this script is that we're going to perform the same action over a list of items. Now, this is the perfect time to use a for-loop. Let's take a second and look at the for shell built-in. The type -a command says four is a shell keyword. So that's a shell built-in, and so what we can do is get help for for. I know there's going to be a good bit of information that pipes out to the screen. So I'm just going to pipe this to the Head command to get just the first 10 lines of output. Okay, the first line of the help says, for name and words, do commands, done. The name you see in the help here is actually a variable name. For each item in the list of words, the variable name is set to that item, then the commands are executed and the process is repeated for each item in the list. Let's create a loop right here at the command prompt. We'll do four X, X is the name or our variable name. And then we're going to provide a list. We'll do this, Frank, Claire and Doug. The next thing want to do is type in do, hit Enter, and then it's stylistically appropriate to end dance. So I'll just put a couple of spaces here, echo, type hi, and then I refer to the variable name here, dollar sign X and hit Enter and close our four-loop with done. In this example, X is first assigned to Frank, then the command of echo gets executed using that variable dollar sign X. Then the next time the loop repeats, Clare is assigned to X, the commands execute, and finally, Doug is assigned to X and the commands execute. If I hit the Up Arrow here to pull up the command, you can the semi-colons which represent the command separators. So if you want to, you can place this command all on a one line, but it's easier to read if you do not do that. Again, it's valid syntax here like you see at the help for on your screen. But again, it's just easy to read if you put these on multiple lines. You want to hit Enter again to execute that again. If you look near the top of your screen in the help output for the four key word, it says, if in words is not present, then dollar sign at sign is assumed. So let's find out what dollar sign at sign is by looking at the Bash Man Page. I'm going to look for the special parameters here in the Man Page. Here's the section. Now let's start with the at sign special parameter. It expands to the positional parameters starting from one. When the expansion occurs within double quotes, each parameter expands to a separate word, that is, dollar at sign is equivalent to $1, $2, and so on. Now this is nearly the same as dollar sign asterisk. It also expands to the positional parameter starting from one. However, if it's enclosed in double quotes, it expands to a single word. We don't want to treat all the parameters as a single word. What we want is a list of words as they were supplied on the command line. That's why we're going to use dollar sign at sign and not dollar sign asterisk in our script. By the way, the use of dollar sign asterisk and dollar sign at sign are often confused. Okay, let's get out of the Man Page here with Q and get back to editing our script. So we're going to use a four-loop. We'll do for, we'll supply a variable name. We'll call these usernames, user underscore name and we'll use quotations here, dollar sign at sign, do. So we're going to use the command we used to generate passwords in the previous script. So we'll do that with password equals, we'll use command substitution with dollar sign followed by an opening parentheses. And then we're going to display that username and password. Now obviously, we can't know exactly how many positional parameters a user will attempt to supply at the command line. We can check for them, but we just don't know in advance. So instead of generating a password for $1, $2, $3 and so on, even though they may only supply two arguments, for example, then what we're going to do is use dollar sign at sign. So no matter how many arguments they pass into the script, it is covered by dollar sign at sign. So we don't have to guess what the user is going to do. They can supply one, two, three or 10 or more and this script is going to work just fine. We'll just supply one thing on the command line to see what happens. So sure enough, the for-loop executed one time, generated a password and displayed it for the user. Let's add another user on the command line. Let's add some more actually. So by using the for-loop in conjunction with dollar sign at sign, then we were able to account for everything passed in on the command line. Just for demonstration purposes, I want to show you what the difference is between dollar sign at sign and dollar sign asterisk. So let's just go back into our script and make that change. So here I'm going to change the at sign to an asterisk, save our changes, and then execute our script again. So as you can see, it treated every thing on the command line as one argument. So the for-loop only ran one time and generated a password for that one argument. Again, that's the difference between dollar sign asterisk and dollar sign at sign. Let's go back and fix our script here and change this asterisk to the at symbol here, save our changes and check our script and sure enough, it runs like we want it to. Let's take just a minute to recap this lesson. And by the way, we covered a lot here. First, you learned about positional parameters and how dollar sign zero expands to the name of the shell script and dollar sign one expands to the first argument supplied on the command line, dollar sign two expands to the second argument, and so on. You also learned a few different special positional parameters, such as dollar sign pound sign which represents the number of parameters, dollar sign asterisk which expands to the list of positional parameters starting at dollar sign one and dollar sign at sign, which also expands to the list of positional parameters starting at dollar sign one. However, the difference between dollar sign asterisk and dollar sign at sign is only noticed when enclosing them in quotation marks. That difference is that dollar sign asterisk treats all the positional parameters as a one string and dollar sign at sign treats them as a separate words. We also took a slight detour in the middle of the lesson to talk about how the path shell variable is used to store a list of colon-separated directories that make up the command search path. You also saw how the path variable can affect what commands get executed and why that is. Along the way, you were introduced to the Witch command as well as the Basename and DIRname commands. Finally, you learn how to create a for-loop, both at the command line as well as in a script.
Jason is the founder of the Linux Training Academy as well as the author of "Linux for Beginners" and "Command Line Kung Fu." He has over 20 years of professional Linux experience, having worked for industry leaders such as Hewlett-Packard, Xerox, UPS, FireEye, and Amazon.com. Nothing gives him more satisfaction than knowing he has helped thousands of IT professionals level up their careers through his many books and courses.