The course is part of this learning path
This training course provides you with a deep dive into how to refactor and structure your code into smaller more manageable building blocks using Functions, Modules, and Packages.
- Understand how to write and define functions
- Understand the four kinds of different function input parameters and how to use them
- Review how modules are created and used
- Learn how modules are loaded using the import statement
- Examine how modules are discovered via module search locations
- Review how modules can themselves be organized into packages
- And finally, we understand how to use aliases for both module and package names
- A basic understanding of the Python programming language
- A basic understanding of software development
- A basic understanding of the software development life cycle
- Software developers interested in learning how to write Python code in a Pythonic way
- Python junior level developers interested in advancing their Python skills
- Anyone with an interest in Python and how to use Python to write concise and elegant scripts for general purpose tasks
About the Author
Jeremy is the DevOps Content Lead at Cloud Academy where he specializes in developing technical training documentation for DevOps.
He has a strong background in software engineering, and has been coding with various languages, frameworks, and systems for the past 20+ years. In recent times, Jeremy has been focused on DevOps, Cloud, Security, and Machine Learning.
Jeremy holds professional certifications for both the AWS and GCP cloud platforms.
- [Jeremy] Okay, to begin with, let's talk about functions. Functions are a way of isolating code that is needed in more than one place by refactoring it to make it much more modular. They are defined with the def statement or def keyword. Functions can take various types of parameters as we'll see in the following slide. Parameter types are dynamic. Functions, more often than not, can return one object of any type using the return statement. If there is no return statement, the function simply returns None. Let's take a quick look at a simple function using our Python interpreter. We'll take the following code and we'll spin this up. Okay back within our terminal, we'll start the Python3 interpreter. And this time we'll use the def keyword to define our function. We'll call it, say, underscore hello. We'll take zero parameters. And all we'll do is simply print out the statement or print out the string hello, world. Enter. Followed by another call to the print function. Let's print out an empty line. Okay, we can call our function now by using the function name. Enter. And so this function has just been executed, and we see that indeed hello, world is being printed out. We could establish a variable to hold this function. And when we take a look at the type of our new variable, we see that it's set to NoneType. So the reason for this is that when we created our function, we didn't specify a return keyword on it, so it's not returning anything. So we could also check that hello world is in fact None, and you see that it is set to True. Okay, let's now create a new function. Again we use the def keyword. This time we'll call it get underscore hello. Okay, then we'll take zero parameters. And this time we'll return, so we're using the return keyword, then we're going to return a string, which will be hello, world, exclamation mark. Okay, so the key point of this second function is that it's now using a return statement. And most functions that you create will in fact use a return statement to return something. We could again create a variable called hello and set it to be the return value of our get underscore hello function. This time if we print hello, you can see indeed it's printing our result. And then if we check the type of hello, it's now classified a a string which we would expect. And finally if we check to see whether hello is None, it's now False because it's actually a string. Okay, let's create a third function. This one we'll square root a number, and we'll be passing the number in as a parameter to this function. We'll return the square root of this number, so we take our input parameter. We can now perform square roots on different numbers that we pass in, say that we'll call our square root function with the number one, two, three, four. Likewise we can also do it on another number, this time we'll do the square root of two. If we take a look at m, we can see here that the square root of 1,234 is this value here. Likewise if we look at n, we can see that the square root of two is this value here. Finally let's use a string format to print out our values, m and n, to three decimal places. Okay, so that was a quick introduction to functions. Let's now carry on. Okay, so as we've briefly just seen, functions can have input parameters. Functions can accept both positional and named parameters. And furthermore, parameters can be either mandatory or optional. They must be specified in the order presented in the next slide. The first set of parameters, if any, is a set of comma-separated names. These are all required. Next, you can specify a variable preceded by an asterisk. This will accept any optional parameters. After the optional positional parameters, you can specify required named parameters. These must come after the optional parameters. If there are no optional parameters, you can use a plain asterisk as a placeholder. And finally you can specify a variable preceded by two asterisks to accept optional named parameters. Okay, let's jump back into our Python interpreter and take a look at each of these different types of function parameters. Okay for our first example, we'll define fun underscore one which will take no parameters. And we'll simply print out hello world again. Okay, as we know we can call this function like so. Now what happens if we attempt to pass in a parameter? It would tell us that it takes a zero positional arguments but one was given. Okay, next we'll define fun underscore two which will take one required parameter, n, and will simply return n squared. Again we can call it. We'll pass in, pass in five. Five squared is 25. Now if we call it without a parameter, we'll get an error saying that one required positional argument is missing. Okay, in our next example we'll define fun underscore three and it will have a required parameter with a default value, in this case we'll call it count, and we'll set the default value to be three. We'll then loop over this using the range function over count. And we'll print spam comma end equals. Enter. So now if we run it, we'll call fun underscore three with no parameters, we get spam written out three times because of our default value for count. And this time we'll pass in 10 for count, and this time we get spam written out 10 times. So that is parameters that have default values. Okay, in our next example we'll do fun underscore four, and this will be defined to have one fixed plus also optional parameters. We do that by specifying firstly our fixed parameter. And then using the asterisk symbol, we can state that we can have optional parameters afterwards. We'll then use the following print statements. And now we can call it. So we'll call it fun underscore four with the value apple. And you can see that n is indeed apple, and that our optional variable is actually a tuple. So let's call it with some extra values. Again we can see that our first parameter is fixed, that's set to apple, and then our optional parameters have been captured in a tuple. Okay, in our next example we'll create another function, and this one will be called fun underscore five, and this will be designed to have keyword only parameters. So we put in a placeholder first and then we'll have two parameters. We'll declare this function to have the following print statements. And now we can call it like so. So we can go fun underscore five, and we'll specify an input parameter called spam with the value equal to one, and eggs equal to two. Okay, we can call it again. This time we can change the positions of these parameters, and we'll see that we'll get the same result. We can call it by passing in spam only and leverage the fact that a default value is set on eggs. We can do the same with eggs. And spam comes out with its default value. And then finally we can call that with no parameters, and we get both default values coming out. Okay, in this next example we'll define another function called fun underscore six, and it will use keyword named parameters. So we do so by using double asterisk named underscore args, and we'll declare it like so. We can then call this function, passing in our named keyword parameters. So the first one will be name equals. Quest equals grail. And color equals red. Enter. So we can see there that our named arguments have come through. The first one with the name set to the value lancelot, second one, quest, set to the value grail, and third one, color, set to the value red. Cool. Okay, let's just redefine our last function. And this time what I wanna do is I wanna print out named args as well as the type of named args. Okay, we'll just recall it like this. So what you can see here is that our named args is actually passed into our function as a dictionary with these key-value pairings. So as we've just seen in our examples, functions can have default parameters, required parameters which can have default values. They are assigned to parameters with the equal sign. Parameters without defaults cannot be specified after parameters with defaults. Okay, let's take a closer look at default parameters. We'll take the following example and run it within our Python interpreter. So for starters let's declare our spam function like so. Here we can see that we have two positional parameters, greeting and whom. Greeting doesn't have a default value, and whom does which is set to world. We can then call spam like so. So whom is taking on the default value, and if we call it again but this time we pass in Jeremy, whom is set to take this value here. Okay, in our next example we'll define the ham function, and we'll then call it like so, ham file name equals. So we can see here that it's executed and that file format has taken on its default value. And then if we call it a second time, this time we'll pass in file name and file format, we get the expected result. Now what happens if we were to call the function like so? As expected, this fails because it's now attempting to pass in positional arguments, but the design of the function has an asterisk, meaning that there are no positional arguments. Okay, next we'll talk about name resolution or what's otherwise referred to as scope. A scope is the area within a Python program where an unqualified name can be looked up. Scopes are used dynamically. And there are four nested scopes that are searched for names in the following order. Local, local names bound within a function. Nonlocal, local names plus local names of outer functions. Global, the current module's global names. And builtin, built-in functions. Within a function, all assignments and declarations create local names. All variables found outside of local scope, that is, outside of the function, are read-only. Inside functions, local scope references the local names of the current function. Outside functions, local scope is the same as the global scope, the module's namespace. Class definitions are also created in local scope. Class definitions also create a local scope. Nested functions provide another scope. Code in function B which is defined inside function A has read-only access to all of A's variables. This is called nonlocal scope. Let's now take the following example, and we'll run it up in our Python interpreter to see the different types of scope in action. Okay, we'll start up our Python3 interpreter. And in this example we'll start off with defining variable x which will be considered to have global scope. And then we'll define a function, function underscore a like so. So function a has variable y as a local scope variable, set to five, and then it has a nested function called function underscore b. Function underscore b has its own local scope, the variable z equal to 32. We then return function b as part of function a's definition. So now we can call it like so. We could set up another variable called f and set it to be function underscore a. So type of f is a function as expected. So now we can actually call f. So here we see the results. So function b gets access to z which you'd expect because z is locally scoped to function b. Function b also has access to y because function b is nested within function a. And function b can also see the global variable x, x is 42. And then the final call here is using builtin scope, allowing the type function to be called on x. Okay, we'll now focus this particular part of the discussion on the global statement. The global keyword allows a function to modify a global variable. This is universally acknowledged as a bad idea. Mutating global data can lead to all sorts of hard to diagnose bugs because a function might change a global that affects some other part of the system or program. It's better to pass data to functions as parameters and return data as needed. Mutable objects such as lists, sets, and dictionaries can be modified in place. The nonlocal keyword can be used like global to make nonlocal variables in an outer function writable. Again we must emphasize that using globally scoped variables is dangerous and should be avoided as much as possible. Okay, we'll next move on to talking about modules. A module is a file containing Python definitions and statements. The file name is the module name with the suffix .py appended. Within a module, the module's name is available as the value of the global variable name. To use a module named, for example, spam.py, then we would import the value spam. This does not enter the names of the functions defined in spam directly into the symbol table, it only adds the module name spam. Use the module name then to access the functions or other attributes. Python itself uses modules to contain functions that can be loaded as needed by scripts. A simple module contains one or more functions. More complex modules can contain initialization code as well. Python classes are also implemented as modules. A module is only loaded once even when there are multiple places in the application that import it. Modules and packages should be documented using docstrings. When working with modules, you need to use the import statement. The import statement loads modules. There are three variations, import module, from module import function-list, and from module import asterisk. We'll now take a closer look at each of these three variations. Variation one. Import module loads the module so its data and functions can be used, but does not put its attributes, names of classes, functions, and variables, into the current namespace. Variation two. From module import function imports only the functions specified in the current namespace. Other functions are not available even though they are loaded into memory. And variation three. From module import asterisk loads the module and imports all functions that do not start with an underscore into the current namespace. This should be used with caution as it can pollute the current namespace and possibly overwrite builtin attributes or attributes from a different module. Let's now create our first module. We'll take the following Python code and create a module called spam. Okay, back within our terminal, we can see that we're in the current directory called PythonDemo. Let's do a directory listing. And as you can see it's empty. We'll use Visual code to set up a couple of files in this directory. The first file we'll add we'll call samplelib.py for Python. And in this file we'll add the following statements. So we'll declare three functions. The first one called spam, the second one called ham, and the third one called underscore eggs. Note the underscore here. We'll come back to this later. We'll save that. And then we'll create a second file, this one called samplelib1.py. Okay, in this Python file we'll use the import statement, and we'll import samplelib like so. We'll use the print function just to specify that everything is okay. And then we will attempt to call our module functions declared in samplelib. The first one was ham, and the second one was spam. We'll save that. Okay, we'll jump back to our terminal. We'll do another directory listing. And now we'll use python3 to execute our samplelib1 file. And there we go. So we can see that indeed our functions declared in our samplelib module have actually been called. So, great result. Okay, let's return to Visual code. And this time we'll update our import statement like so. From module name import spam and ham. We can then go down to where we call these functions and remove samplelib from them. By taking this course of action, we import functions, spam and ham, from the samplelib module into the current namespace, meaning the module name is no longer required to call the functions. So we'll save this file. We'll go back to PythonDemo, and we'll rerun the script. Again that's worked as expected. Returning to Visual code, we'll create a third file called samplelib2.py. And this time we'll use the following type of import statement. So from samplelib import asterisk for everything. We can then call spam and ham again. Again we can do so without explicitly referring to the module name. Back to our terminal, we'll run samplelib2, and again it works as expected. Okay, returning to Visual code one last time, we'll create a third file called samplelib3.py. And this time we'll use the following import statement where we're aliasing our functions. So spam is being aliased as pig, and ham as hog. We can then refer to these functions by their aliases, pig and hog. Save. We'll return to the terminal. And we'll run this, and again it has worked with the same results. Now it must be noted that importing everything through the import asterisk statement should be considered dangerous. Using import asterisk to import all public names from a module has risk with it. While generally harmless, there is always the chance that you will unknowingly import a module that overwrites some previously imported module. To be 100% certain, always import the entire module or else import names explicitly. We'll demonstrate this with the following example. So back within Visual code, we'll create a new file called electrical.py, and we'll give it the following Python code. So in here we have three functions, amps, voltage, and current, and then each of those just returns a value defined in global scope up here. Okay, we'll save that. We'll create a second file called navigation.py. Navigation will contain the following code, noting that it also has a function called current. So in electrical we define current to be this which returns the default underscore current value defined here. However now in navigation we also are defining current, but this time current will return the first item, slow. And then lastly we'll create a third file called why_import_star_is_bad.py. And in this file we'll import from electrical everything, and likewise from navigation. We'll then add some print statements. So we'll call current, we'll call voltage, and we'll call amps. So the intention here is that we're calling each of these three functions from our electrical module. However because we have imported everything from navigation, and navigation also defines a current function, then when we go to run this, we'll now run our file, python3 name of the file. In here you can see that we've got slow, 110, and 10. So slow is the result of current coming from navigation, not from electrical. So if we look at navigation again, current is returning current types, and current types is defined to be the split of these words. So that's why we see slow when actually our intention was to return current which would have been this value here. Okay, let's now talk about how modules are discovered at runtime. When you specify a module to load with the import statement, it first looks in the current directory, and then searches the directories listed within sys.path. To add locations, put one or more directories to search in the PYTHONPATH environment variable. Separate multiple paths by semicolons for Windows, or colons for Unix/Linux. This will add them to sys.path after the current folder but before the predefined locations. The following example here sets PYTHONPATH for Windows whereas this example is for Linus and or OS X. You can also append to sys.path in your scripts, but this can result in non-portable scripts and scripts that will fail if the location of the imported modules change. It is also sometimes convenient to have a module as a runnable script. This is handy for testing and debugging, and for providing modules that can also be used as standalone utilities. Since the interpreter defines its own name as underscore underscore main underscore underscore, you can test the current namespace's name attribute. If it is underscore underscore main underscore underscore, then you are at the main or top level of the interpreter and your file is being run as a script. Any code in a module that is not contained in a function or method is executed when the module is imported. This can include data assignments and other startup tasks, for example, connecting to a database or opening a file. Let's now take a look at an example where the Python script checks to see if it is the top level script, and if it is, it calls the main function within itself. Back within Visual code, we'll create a new file called main.py, and this time we'll paste the following code. Now the key point about this file is that at the bottom of it, we're checking to see if it was the file that was used to start up the program, and if it was, we then call the main function, which is very much a typical naming convention for our first starting function. So we'll save this, and then we'll jump back to our terminal to our directory listing. So we have main.py, and this time we'll do python3 and the name of the file. And there we go. So the file has executed successfully. Okay, we'll now move on to a discussion around packages. A package is a group of related modules or subpackages. The grouping is physical, that is, a package is a folder that contains one or more modules. It is a way of giving a hierarchical structure to the module namespace so that all modules do not live in the same folder. A package may have an initialization script named underscore underscore init underscore underscore.py. And if present, this script is executed when the package or any of its contents are loaded. Modules in packages are accessed by prefixing the module with the package name using the dot notation used to access module attributes. As an example, if module eggs is in package spam, then to call the scramble function in eggs, you would likely call spam.eggs.scramble. By default, importing a package name by itself has no effect. You must explicitly load the modules in the packages. You should usually import the module using its package name, like from spam import eggs, to import the eggs module from the spam package. Packages can be nested. Let's now take a look at a package example. In this example we have the sound package which is considered the top-level package. Beneath this, we have a initialization script for the sound package. Then we have two subpackages, the file format subpackage and the sound effects subpackage. Each of these also has its own initialization script. Stored within the same sound package, we have an additional filter subpackage, again it also has an initialization script. Then at the bottom, we can now see the import statements that are required to import the package and modules. For convenience, you can put import statement into a package's initialization script to autoload the modules into the package namespace. So having now reviewed functions, modules, and packages, the following next two slides show you all of the various types of import statements. Take time to review this. Take time to understand what the import statement is and what it achieves. Okay, moving on. Documenting modules and packages. In addition to comments, which are typically used by the maintainers of your code, you should also add docstrings which provide documentation for the user of your code. If the first statement in a module, function, or class is an unassigned string, it is assigned as the docstring of that object. It is stored in the special attribute, underscore doc underscore, and so is available to code. The docstring can use any form of literal string, but typically triple double quotes are preferred for consistency. See PEP 257 or Python Enhancement Proposal 257 for a detailed guide on docstring conventions. Tools such as pydoc and many IDEs will use this information. Okay, the following two slides will show you examples of using docstrings. Here in this slide, we can see that above the import sys statement is a docstring which documents the intent of this module. Likewise in this example, we have a main function in a function1 function. Inside each of these functions, docstrings are used to describe the intent of each function. Keep in mind that when you're writing your Python code, that you should do so using a Python style. On this slide are a number of guidelines that should be read and understood and applied to make sure that your code, when written and developed, remains Pythonic. There are many resources on the internet that will help you to write Pythonic styled code. Take time to read both the Python Enhancement Proposal 8 which is the Style Guide for Python Code, and also the Python Enhancement Proposal 257 which documents Docstring Conventions.