When most people think about managing files on a computer, it's a GUI interface like Gnome Nautilus that usually comes to mind first. However, while navigating through your files visually does certainly have some benefits, once you get used to the speed and power of command-line file management you may never want to go back. But in any case you'll have no choice but to use these skills when you're logged into a headless server, remote machine, or LXC vm. So consider these commands to fall pretty much in the "basic survival skills" category.

Basic Linux file and directory management

Let's start at the beginning with some commands that we've come across already in these courses. We'll begin by listing the files and directories in our current location using "ls." You can create a new empty file using touch followed by the file name. If a file with that name already exists, then touch will simply update its last updated information. You can create a new directory using mkdir. If you want to create a directory somewhere on your file system besides the current location, simply include the absolute address in your command, and if you'll need admin rights, sudo. You'll sometimes need to create an entire directory tree. This is common when you're installing a package that expects a data file to exist in a certain location whose parent directories don't yet exist on your system. This requires just one simple command using mkdir with the "-p" switch. See how long it would take you to do that in the GUI file manager.

Let's return to our home directory. You can copy files or directory trees between locations using "cp." "Cp file1 newplace/" will make a copy of file1 in the new place directory. If you want to copy a directory and its file and subdirectories all at once, we would add "-r" to the "cp" command to make the operation recursive. Let's go take a look to make sure both the directory and the copy of the file called "file1" have been copied faithfully. Remind me to clean up that poor /etc/perl directory once we're done with all this.

You can move a file or directory tree from one place to another or just rename them using "mv." Where "file1" is the original name of the file we'd like to change and "file2" is the new name we'd like it to have. Since we've been using "ls" to list the contents of our files a bit, now's probably a good time to show how using "ls -l" you can list files and directories with longer descriptions. And running "file" against a file directory will display useful file information.

"Rm" will delete or remove a file. "Rm -r" followed by the name of the directory will remove the directory and all the files and directories that it contains. Be aware, be very aware, that "rm" by default will not ask you if you're sure you want to remove these files and it won't place the files in a trash can that can be recovered later. Once you run "rm," your files are for all intents and purposes gone. Having said that if you run "rm" with the "-i" switch, you'll be asked if you're sure before anything happens.

One of the things that really makes the command line stand far apart from GUI file managers is globbing. Even if the word itself sounds like it might be referring to some mildly toxic liquid that oozes from between the keys of your keyboard, it's actually about adding global identifiers to your commands.

To illustrate let's create a few new files to replace the ones we've just removed. Now let's use "ls" and "grep" to list all the files in this directory whose names begin with the letters "F" and "I." If we now run "rm file" with a question mark, it will delete file 1 and file 2 but not file 22. Why not? Because the question mark will include exactly one instance of any character. If you would have wanted to remove file22, which has two characters after the word file as well we would have used an asterisk instead.

This kind of pinpoint accuracy in identifying files is obviously going to be useful far beyond simple file deletion but can also play a huge role in sophisticated scripting.

Let's take a moment and review what we've seen. Touch will create or update a new file. Mkdir will create a new directory. "Mkdir -p" will add any parent directories that may be necessary. "Cp" creates copies of a file and "cp -r" copies directories and their contents. "Mv" moves or renames files or directories. And "rm," "rm -r," and "rm -dir" remove files and directories. The question mark when used for file globbing will include any single character while an asterisk will include any number of characters.

Archives and file compression using tar

Your lives as Linux system administrators will never be complete unless you can create archives and backups of large groups of files and directories. Now that we're able to use tools like globbing to identify specific subsets of file systems, we should also be able to pipe the lists of files to generate entirely new backup archives. Tar, which stands for Tape Archive, although it obviously works just as well without backup tapes, uses existing archiving and compression software to create, amend and restore file archives. "Tar-cvf" where c stands for create, "V" for verbose and "F" for the name of a file will create a new archive called newarch on the USB device indicated. The asterisk tells tar to create that archive from all the files but not directories in the current directory.

An archive is simply a structured collection of files that are bundled into a single file. Adding the letter z will tell tar to compress the archive as well.

An uppercase A will append rather than create an archive. By the way, tar always expects the name of the new archive file you want to create to immediately follow the "f" switch. And the files or directories to be included to follow after that. This can be a bit confusing but you absolutely must remember this detail for the LPI exam. "-x" used this way will extract the files of an archive into the current directory. As you can see, tar archives take a -tar file extension, and archives that were compressed using gzip will end with tar.gz.

Besides working through tar, you can also directly compress or decompress archives using the gzip, gunzip, bzip2 or xz programs. You'll probably run into archives in these formats during your career and once again the LPI exam expects you to be familiar with them.

Archives and file compression using cpio

The cpio utility does a lot of what tar can do although it usually works with pipe data rather than as a standalone. Let me show you what I mean. Let's list all the files in this directory. There's currently only one actual file besides the directories. Now let's list the files and pipe the list to cpio. "-o" will create the archive called "lsarch.cpio." Adding gzip to the command will compress the archive. Now let's move to the root directory and use the find tool to search to a maximum depth of four directories for any file with a file extension of .txt. There are a couple of such files. Now let's use cpio to create a new archive of just those files. We'll just jump back into our home directory to make sure the archive has in fact been created.

Finally, we must discuss "dd," which some believe stands for "disc destroyer," since it's so easy to misuse it to wipe out vast volumes of valuable data. Still dd's great risk is more than balanced by its still-greater value. Dd always takes an "if-of" structure. That is, "if" equals the files I'd like to copy and "of" equals the place to which I'd like them copied. The beauty of dd is that it will copy a file system or an entire partition in its exact original format. It can use this to copy an entire hard drive and move it over to a new computer. And you'll be able to boot from it as though it was the original. Here's a simple example where the goal is to copy the sdb1 partition referenced by the link the /dev directory to the location of your backup media. This will copy a whole drive sdb to the backup drive you've mounted in mnt/drive. We're not going to talk about it in this video but adding tools like SCP and Rsync to the mix will allow you to perform all of these functions not just locally but between remote computers often all using a single command.

So let's review all of this. Tar with the "-c" switch will create a new archive. Adding a z will compress it too. And uppercase A will append new files to an existing archive. Tar-x will extract the contents of an existing archive. cpio when you pipe data streams to it, will create archives and when gzip is added these archives will be compressed. You can use the powerful find tool to carefully select which files to pipe to cpio or to any one of dozens of other Linux command-line tools for that matter. And dd will copy an exact image of a partition, drive, or selection of files.