Linux Programming Conventions

Advanced Standard Input, Standard Output, and Standard Error - Part I
Overview
Difficulty
Intermediate
Duration
51m
Students
96
Ratings
4/5
starstarstarstarstar-border
Description

In this course, you'll learn about the different types of input and output and how to redirect and control those types of input and output. We'll then walk through an exercise that focuses on creating users on a Linux system - this exercise builds upon the exercise that we did in the Password Generation and Shell Script Arguments course. 

This course is part of the Linux Shell Scripting learning path. To follow along with this course, you can download all the necessary resources here.

Learning Objectives

  • Learn about the different types of input and output and how to redirect and control those types of input and output
  • Gain practical experience of how to create users on Linux

Intended Audience

  • Anyone who wants to learn Linux shell scripting 
  • Linux system administrators, developers, or programmers

Prerequisites

To get the most out of this course, you should have a basic understanding of the Linux command line.

Transcript

In this lesson you're going to learn about the different types of input and output and how to redirect and control those types of input and output. So as always, I have a local terminal up on my machine and I'm going to move into our working folder for this class and into the project folder. Next, I'm going to bring up our virtual machine. Now that the virtual machine is up, I'm going to connect to it. From here I'm going to move into the shared folder of forward slash vagrant. The name I'm going to use for our demo script today is luser-demo08.sh. I'm going to add the shebang here and give a path to the bash executable. And I'll just add a short header to our file. So far in this course, the only type of input and output redirection that you've used has been in the form of a pipe. You've already written scripts that redirect the standard output from one command and send it as standard input into another by separating those commands with a pipe. You've used that method to string together several commands in order to generate a random password for example. You also used it to send that random password to the password command. Let's take a step back and talk about the different types of input and output. First off, there are three types of IO. They are standard input, standard output and standard error. By default, standard input comes from the keyboard. For example, the read command except standard input and you use the read command to collect information from a user typing something in at their keyboard. You also know, however that standard input doesn't have to come from a keyboard. Again, you used pipes to take the output generated by one command and use that as the standard input for another command. Beyond standard input there is standard output and standard error. By default both standard output and standard error are displayed to the screen. Let's start out by redirecting the standard output of a command into a file. And you can do that by using the greater than sign. By the way, you'll often see standard out abbreviated as STDOUT. We're going to be using a file several times throughout this script so I'm just going to assign that file name to a variable. I'm going to call that variable file. And then I'm going to say that that file is going to live and forward slash tmp. And we'll call the name of that file data. Let's use a command here that we've already used in a previous exercise. We'll use the head command. - n1 will output the very first line of a file. And we'll just look at the first line of the etc password file. And we'll send the output of this command, the standard output of this command into the file. Now, remember by default standard output is displayed to the screen, but since we're using the greater than symbol after the command it is redirecting that output away from the screen and into the file we specified after the greater than sign. So let's see what happens when we actually execute this script. So I'm going to save my changes, make sure that this is executable. And then execute the script. As you can see no output was generated by our script and that's pretty much to be expected because we told the output of the head command that's in the script to go to a file. Now let's look at that file. Okay. That seems to be the output of head-n1 etc password. And we can actually confirm this by just running the command here on the command line. And sure enough that is the output of the command that's in our script. And that output was not displayed to the screen, but instead went to the file where we redirected that output. To be super clear, this output redirection is not part of any one command. It works with all commands. While we're on the command line, let's try it out with, let's say the ID command. So we'll use id-un. We've used this in a previous project. And we'll send the standard output to a file that we'll decide to call id. Again this file name could be anything. So no output goes to the screen. However, if we look at the contents of the id file with a cat command, we see the output that would be generated by that command. Again, we can do this with any command, let's use echo. And if we cat the uid file, sure enough, the output of the echo command is in the file that we redirected that output to. By the way file permissions are in play here too. So if you try to redirect output into a file where you don't have write permissions, you're going to get an error. So let's back up here with our echo command and let's try to write this and say forward slash uid and hit enter. And sure enough, we get a permission denied error. Let's look at the permissions on forward slash. They're set to 555 or read execute, read execute, read execute. We're the owner of root and the group of root. Well we're vagrant and we're in the group of vagrant, so we have no write permission. So we can't create files there. We can't do that either through redirection or using an editor or any other means. We just simply don't have permissions. I just wanted to point out that if you get a permission denied error, when you're working with redirection, take it at face value. You don't have permissions there. Okay, let's jump back into our script. Okay, now we know that the greater than sign redirects standard output. Well, to redirect standard input use the less than sign. Again, standard input is often abbreviated STDIN. Let's use an example with the read command. We'll do read line, do an input redirection and then supply a file. And we'll just use the same file that we created with the head command. Now you've used the read command to accept standard input from a user. By using input redirection you are now accepting standard input from a file. The read command reads one line of standard input. When someone is typing at the keyboard and they hit enter that represents one line of standard input. Of course, when dealing with files a new line character will call us read to stop. So the variable line should contain the first line of the file. In this example, there is only one line in the file. Let's prove that by echoing it to the screen. We'll say that the line variable contains the contents of the line variable here and hit enter. Let's save our changes and execute the script. Stepping through this short script we used output redirection to send the first line of the etc password file into a file named /tmp/data. That file was then used as standard input to the read command which assigned the value to the line variable. Finally, the line variable was displayed to standard output using the echo command and that gives you what you see on your screen here. Redirecting standard input from a file works with any command that except standard input. Take the password command as an example. Let's put some text into a file. And we can do that with the echo command. we'll just echo out some information here. Normally that would be displayed to our screen but we're going to use output redirection to put that output in a file called P-A-S-S-W-O-R-D. So we'll look at the contents of that file. Sure enough, it contains the word secret. Now let's use that file as standard input to the password command. So we need root privileges to do this. We're going to use sudo password with a standard in option and we need to supply a user while I have the Einstein user set up on this system. So I'm going to use that user and then I'm going to redirect in the standard input that lives in the file password. So we're going to redirect the contents of the file name password as standard input into the P-A-S-S-W-D command. Here it says the passwords were changed for the user Einstein and I can verify this by just switching users. And I'm going to enter S-E-C-R-E-T and hit enter and sure enough, it accepts the password. So that works. So the first method you used was with a pipe but now you can also use this input with the less than sign. And remember the difference between those methods is that in a pipe the standard input comes from the output of a command whereas with this less than symbol the standard input comes from a file. So that's the difference. You use a pipe when you want to take output from a command as input into another command but you can use this redirection when you wanna redirect the contents of a file into a command. It's important to know that using a single greater than sign to perform a redirection creates a new file if that file doesn't exist, or it overwrites the contents of that file if it does exist. So right now the password file contains the word secret. If we perform a redirection to that file, it will get overwritten. So we'll do this. We'll just echo the word new into the password file. So now when we look at the contents of that file, it contains new. Now let's demonstrate this in our script. So now we're going to take the first three lines of the etc password file and send that into this file. Going to echo a blank line to the screen and then now show the contents of that file. Okay, let's save our changes and execute the script. The first time we performed a redirection to the script we took the first line of the etc password file and put it into a file. The next time we performed redirection to that file we took the first three lines of the etc password file and wrote it to that file. So here you can see that the contents of tmp data contains three lines, not four lines. It didn't append to it, it overwrote the contents of that file. So what if you don't want to overwrite a file but you want to add or append to it. In that case you use double greater than symbols. So let's look at the contents of our password file here in our current directory. It contains just one line and that line says new. Now let's add another line to the file. We'll do this echo another line. And then we'll use two greater than symbols. And by the way, they can't have a space between them. They should be together. And we're going to provide the name of that file that we're going to redirect this output into. The name of that file is P-A-S-S-W-O-R-D. So now when we look at the contents of this file, we should see two lines. And sure enough we do. The original contents, which was new and then the new contents, which is another dash line. Let's keep adding to this file by using one of our password generation techniques. We'll just take the current date, take the output of that as input to the sha256 sum command. And then we'll just take the first 10 characters that that command generates. And then we'll append it to the P-A-S-S-W-O-R-D file. Now the file contains that extra data. Now, I think it's pretty darn awesome how you can combine different types of input and output redirection all on the same command line. So here we use pipes and output redirection. Let's add this as an example of IO redirection to our script. So now we're going to redirect standard out to a file appending to that file. So let's just echo a couple of random numbers in there. And then we'll just repeat this line. Oops, I missed a dollar sign in front of this variable. I'm going to place it there. And now I'll just add a blank line and then echo the contents of that file. Let's save our changes and execute the script. Okay, so originally the contents of tmp data was one line. It was overwritten by three lines and then we performed an append output redirection. And so now that has five lines. You can see the random numbers that we generated and appended to the file. So we've covered redirecting standard input and standard output.

About the Author
Avatar
Jason Cannon
Founder, Linux Training Academy
Students
3453
Courses
61
Learning Paths
8

Jason is the founder of the Linux Training Academy as well as the author of "Linux for Beginners" and "Command Line Kung Fu." He has over 20 years of professional Linux experience, having worked for industry leaders such as Hewlett-Packard, Xerox, UPS, FireEye, and Amazon.com. Nothing gives him more satisfaction than knowing he has helped thousands of IT professionals level up their careers through his many books and courses.

Covered Topics