To begin with, let's review what metaprogramming is, in a more general sense. Metaprogramming is writing code that generates or modifies other code. It includes fetching, changing or deleting attributes and writing functions that return other functions. Now, metaprogramming is much easier in Python than many other languages. Python provides explicit access to objects. Even the parts that are often hidden or restricted in other programming languages. For instance, you can easily replace one method with another in a Python class, or, even in an object instance. 

For starters, let's consider the built-in functions, globals and locals. Globals and locals contain all the variables in a namespace. Globals returns all global objects. Locals, on the other hand, returns all local variables. Now, the globals built-in function returns a dictionary of all global objects. The keys are the object names and the values are the object values. The dictionary is live. Changes to the dictionary affect global variables. And finally, the locals built-in function returns also a dictionary, but this time of all objects in local scope. To give you a clear understanding of the globals and locals built-in functions and what they're capable of, we'll run up the following Python code. 

Okay, back within our terminal, let's spin up the Python 3 interpreter. For starters, we'll import the pprint for pretty printing functionality from the pprint module. Like so. We'll set up a couple of local variables, spam equals, and we'll just give it the number 42, and also ham and we'll set it to Smithfield. We'll now define the following function, eggs. And all eggs is going to do is it's going to have a couple of its own local variables, name and idiom. And then we're going to use a number of print statements to print out what the globals built-in function is and also the locals built-in function. 

Okay, so now if we run eggs, some eggs please. And we can see here that globals has captured a dictionary of this information. And likewise, locals function provides us a dictionary of the locals variables. In this case, we've got fruit, idiom, and name. So, here you can see clearly the features provided by calling the built-in globals and/or locals function. Okay, back to the slides. Working with attributes. All Python objects are essentially dictionaries of attributes. There are four special built-in functions for managing attributes. The four special built-in functions are getattr, hasattr, setattr, and delattr, where actually, the attribute name is abbreviated to A-T-T-R, as can be seen here. The getattr function returns the value of a specified attribute, or none, if the object does not have that attribute. Hasattr returns True if the specified object has the specified attribute, otherwise False. Setattr sets an attribute to a specified value. And, finally, delattr deletes an attribute and its corresponding value. 

Let's now try an example of this. We'll take the following code and run it in our interpreter. For starters, let's define the following class. We'll then create an instance of this class. And we'll call the eggs function on it. Like so. Okay, we'll now interrogate the attributes by calling hasattr. We pass in the instance and the attribute we're after. So, here we can see that there is no attribute called egg. That's set to false. If we re-run this and we do eggs plural, you can see that the attribute does exist. So, now what we can do is we can retrieve this by calling getattr. Pass in our instance and the attribute that we wanna retrieve by name. Now, we can call that attribute. 

So, in this case we'll call it with the input parameter scrambled. And you can see that that function has been called successfully. Okay, let's create another function outside of the class, we'll call it toast, like so. And this time, we'll call the setattr function. Pass in the class name. Pass in the attribute that we wanna set. So, we're going to set eggs and this time we're going to set it to be the toast function, like so. Now, if we call eggs on our instance, and this time pass in the string value buttered, you can see indeed that the function's been swapped out with our updated toast function. And then, finally, what we'll do is we'll use the delattr function. We pass in the class and the attribute that we wanna delete. That's failed because it doesn't exist, it's actually eggs plural. 

Okay, that's now that's been deleted. And this time, let's use a try except block. So we'll try and call eggs again. And if an exception is thrown, in particular, an attribute error exception we'll capture it and we'll print out this error, like so. And indeed, you can see attribute eggs no longer exists because we delete it by calling the delattr built-in function. So, here you can see the power of doing metaprogramming using these four special built-in functions. Let's now consider the inspect module. The inspect module provides user-friendly functions for accessing Python metadata. It helps to simplify the access to metadata. 

Let's now take a closer look at the inspect module by running this code. So, in this example we'll import inspect. We'll create a new class called Spam, again. And we'll just use the pass function to pass on it. We'll then define the following Ham function, which lives outside of the class. Next, we'll define a for loop, like so, to loop through each of these different parts of our program. So, we're going to look at inspect Spam and Ham. And then for each of these we're going to use inspect to ask is this a module or is this a function and/or is this a class? So, let's try this. So here, we can see the output. So, the first one we're inspecting on is inspect itself. Yes, it is a module. No, it is not a function. And no, it is not a class. The next one is Spam, Spam is a class. So, is it a module? No. Is it a function? No. Is it a class? Yes. And then, finally, we're looking at Ham. Ham is a function and as expected, it's not a module, it is a function, and it is not a class. So those are some quite useful functions that can be called upon. 

Next, let's consider the following print function where we're using inspect to call the getfullargspec function on Ham. So the function spec has returned and you can see that there are args, variable args, argument keywords, defaults, and keyword-only arguments. And finally, keyword-only argument defaults and annotations. So again, another very useful function when you're doing metaprogramming. And then finally, let's consider the following print statement, where this time we're calling the getframeinfo function on the inspect module. And you can get information about the current frame at runtime. Okay. So, that is the inspect module, very useful for metaprogramming. As just witnessed, the inspect module provides a number of very helpful and convenient functions for doing metaprogramming. 

The following table specifies a lot of the additional functions that we didn't take a look at, but for which you should review and understand what features are available when using the inspect module for metaprogramming. Let's now move on to decorators. A decorator is a component that modifies some other component. The purpose is typically to add functionality, but, there are no real restrictions on what a decorator can do and can't do. Many decorators register a component with some other component. For instance, the @app.route decorator in Flask maps a URL to a view function. In Python, many decorators are provided by the standard library, such as property or classmethod, with a special syntax. The ampersand, or @, is used to apply a decorator to a function or class. The table provided here on this slide and the next slide provide many of the decorators that are available in the standard library. We won't go through them individually, but do take time to review them and understand each of them and what they can actually do. 

Decorator functions. A decorator function acts as a wrapper around some function. It allows you to add features to a function without changing the function itself. For instance, the @property, @classmethod, and @staticmethod decorators are all used in classes. A decorator function expects only one argument, the function to be modified. It should return a new function, which will replace the original. The replacement function typically calls the original function, as well as some new code. The new function should be defined with generic arguments so it can handle the original function's arguments. And finally, the wraps decorator from the functools module in the standard library should be used with the function that returns the replacement function. This will ensure the replacement function keeps the same properties, especially the name, as the original target function. Okay, let's now take a closer look at wraps by taking the following code and we'll spin it up within Visual Code. 

Okay, back within our terminal. We'll start by doing a directory listing. This time we've got a number of Python files and in particular, we're going to take a close look at this file here. So, we'll start up Visual Code, we'll open up the deco_debug file. And for starters, you can see that we're importing wraps from the functools module. And here you can see we're defining a new function, called debugger, in which the @wraps decorator is being used and is being decorated on a function within this function called new_func. Later on, we have another function called hello, on which we're then applying our custom decorator, the debugger decorator. Okay, let's try this out. 

We'll save this and we'll go back to the terminal. And we'll do python3 deco_debug. And there we go. We can see that we've got our print statements from our custom decorator. Okay, let's move on to decorator classes. A class can also be used to implement a decorator. The class must implement two methods, _init_ and _call_. Init is passed the original function, and can perform any setup needed. The call method replaces the original function. Again, to understand this let's take a look at an example. Again, let's jump back over into Visual Code. This time we're looking at the deco_debug_class Python file. In here our decorator has actually been implemented as a class. We've created the _init_ function and the _call_ function. Again, we apply our custom decorator, the debugger, to our function hello. And then we call hello three times with the following parameters for the first call, likewise the second call, and then the final call. Again, we save this file. 

We'll jump back into our terminal. We then use python3 deco_debug_class. Enter. And again we can see that our decorator has correctly been applied and executed for each call to the hello function. So, that is demonstrating how you would implement a decorator using a class. Again, the key points here is that you need to define the init function and the call function. Decorator parameters. A decorator can be passed its own parameters. This requires some extra work. For decorators implemented as functions, the decorator itself is passed the parameters. It contains a nested function that is passed the decorator function, the target, and that returns the replacement function. Now, for decorators implemented as classes, init is passed the parameters. Call, on the other hand, is passed the decorated function, again, the target, and call returns the replacement function. Again, let's consider another piece of sample code where decorators are being implemented and used. 

In this case we'll consider the multiply function. In this example we're going to look at the deco_params Python file. Again, we're importing wraps from the functools module. And this time, we're creating a multiply function, or a multiply decorator, takes a parameter called the multiplier. We then use the @wraps decorator from the functools module. And then we're updating and defining our new function. And basically, we're taking our multiplier and we're going to multiply it against the result of the original function. So here we can see we're applying our custom multiply decorator. In this case with the parameter 4. This has been applied to our spam function, which is returning 5. So, if our custom multiply decorator has been coded successfully, the value should be returned as 20, four times five. We then have an additional function, again, with the @multiply decorator applied. In this case with a parameter of 10. The ham function returns 8. So, in this case, the return function should end up returning 10 times eight, which is 80. We then call spam and ham and print out the results. 

Okay, so let's see this custom decorator at runtime. Do a directory listing, python3 and we're going to run this guy. And there we go. Indeed, we've got the right result, 20 and 80. So, let's review this again, we'll go back to our code. Five times four is 20 and eight times 10 is 80. So, that's what we get when we call print on a and b. Again, this is just an example of how decorators are implemented. Let's now consider creating classes at runtime. For advanced needs, a class can be created programmatically, without the use of the class statement. The syntax is as can be seen here. The first argument is the name of the class, the second is a tuple of base classes, and the third is a dictionary of the class's attributes. 

Again, to understand how we could do this, let's take a look at the following example back within Visual Code. Okay, in our creating_classes file we've started off by defining two functions, function_1 and function_2. All they do is print out to the console. The real magic in this particular demonstration is this statement here. So what we're doing, is we're creating a new class called NewClass. We do so by using the type function, which is a built-in function. The first parameter for this function is the name of the class, in this case we're calling it new_class. We then pass in a dictionary. The dictionary has key-value items, the first of which is defining our function_1 against the name hello1. Likewise, function_2 is assigned to hello2. We then assign two attributes, color and state, with the values red and Ohio, respectively. We can then instantiate from this NewClass definition. We do so by assigning it to n1. And then finally, we call hello1 and hello2 on our instance. 

And likewise, we take a look at the color attribute. Then, to complete this example, we actually create a second class in which we actually SubClass from the class we previously created. It's a NewClass here, it's provided here. And what we're seeing here, is that NewClass is actually the base class. We then instantiate SubClass and we call hello1 on it and color and fruit. Okay, let's save this file and we jump back to our terminal. Do another directory listing. Again, we run python3 on creating_classes. And everything has worked. So we can see that function_1 was called successfully, as well as function_2, and that we were able to call the color attribute. Likewise, our SubClass has also been instantiated correctly. We can call function_1 from the base class as well as the color and fruit attributes. So again, this shows the power of creating new classes at runtime, programmatically. 

Okay, let's now consider the concept of monkey patching. Monkey patching refers to the technique of changing the behavior of an object by adding, replacing, or deleting attributes from outside the object's class definition. It can be used for replacing methods, attributes, and/or functions, or, modifying a third-party object for which you do not have access to. Or, simply adding behavior to objects which reside in memory. Now, a word of caution, be very careful when creating and applying monkey patches. As often, there are some hard-to-debug problems which can happen. For example, if the object being patched changes after a software upgrade, the monkey patch, later on, can fail in unexpected ways. 

Conflicts may occur if two different modules monkey-patch the same object. And, users of a monkey-patched object may not realize which behavior is original and which comes from the monkey patch. Monkey patching defeats object encapsulation, and, therefore, should be used sparingly. Let's now see a quick example of monkey patching. So in our meta_monkey example, all we're doing is we're starting off with a custom Spam class and the only interesting thing here is that we have a function called eggs, which has a single print statement. 

We then instantiate from our Spam class and we call the eggs function. Next, we create a secondary function outside of the class definition, called scrambled. And we use the special built-in setattr function to basically monkey patch the scrambled function over the top of the existing eggs function within our Spam class through using the setattr built-in function. And then, again, we call the eggs function again. So, the second call will overwrite the original definition of the function. And this, basically, is monkey patching. Okay, save that and we'll go back to our terminal. Directory listing, use python3 and we'll run this guy here. And there we go. So, we can see the original call to the function as it was defined within the Spam class definition. And then a second call to the same function, but after we monkey-patched it. So, quickly reviewing our code, here's our monkey patch and here is where we apply it back into our class definition, overwriting the original definition of the eggs function.

 

Lectures

• Introduction
• Review