This course is the second course in two-part series on how to build an application in Python. In the first course, we built a data ingestion process that extracted named entities from articles across a few different publications. We extracted named entities from around 100,000 articles and we saved the results into Cloud Firestore. In this second course, we'll explore the codebase for a web application used to visualize those results.
We'll kick off the course by checking out some quality of life changes implemented while developing this app. That includes a custom bash theme, a replacement debugger, a debugger command for starting an IPython shell, and pytest plugins. After that, we're going to review the data access layer and its accompanying tests. That's going to include multiple implementations of each data access service. Then we'll check out Python's web application standard.
Next, we'll review the web application layer and its accompanying tests. That's going to include a fast web application framework, custom middleware, request hooks, and application configuration. After that, we're going to review the presentation layer, including a Vue.js app and materialize CSS. Finally, we're going to run the app locally and trace some requests through the application using the debugger.
If you have any feedback relating to this course, feel free to contact us at support@cloudacademy.com≥
Learning Objectives
- Implement a few developer quality of life changes
- Implement a testable data access layer
- Understand how a Python web app operates
- Understand how to build and test a more complex web app
- Understand how to use ipdb and IPython
- Enhance your knowledge of the Python programming language
Intended Audience
This course is intended for software developers or anyone who wants to learn more about building apps with Python.
Prerequisites
- Before taking this course, please make sure you have taken the first course in this two-part series: Building a Python Application: Course One
- You should also have an understanding of Python 3, Linux CLI, HTML/JS, and Git
Resources
The source code for the course is available on GitHub.
Hello, and welcome. In this lesson, we're going to summarize some of the key takeaways from building this application, and some next steps. When I started creating this course, I had three themes in mind. One was developer quality of life. Another was unit testing, and the third was debugging. And the context for recovering all of these themes was our WSGI application.
The application in this course was a fairly minimalist WSGI application. WSGI stands for Web Server Gateway Interface, and it's a Python standard for building web applications and web application servers. The WSGI specification was first published in 2003, and it was revised in 2010 for Python 3. Having the server and applications separate, allows for the creation of reusable server applications, and generalized application frameworks.
We use Gunicorn for the WSGI server, and we use Falcon for the application framework. However, there are different options, each with their own merits. We didn't focus on the WSGI server, though we did talk about how the server expects to be provided with a callable WSGI application.
We reviewed a minimal application that consisted of just one function, and this showed the conceptual ease of a WSGI application. The problem with our minimalist app is that, it couldn't route requests based on the URL. It had no middleware and really no anything.
Frameworks such as Flask, Falcon, Django, et cetera, provide the additional features needed to build out something more complex without having to re-engineer the wheel. Our application was broken into resources. These resources were plain old Python objects, with methods for handling the HTTP verbs that we wanted to support. Each resource accepted its data access dependencies as arguments when instantiating the class.
Then the resources use that data access layer to fetch the data and return the response. The data access layer included both real and fake implementations, which was intended to make it easy to develop and test the WSGI application.
The unit tests were implemented in Pytest, and most of the tests were just basic assertions that when we provide a specific input, that we receive our expected output. To help control the input of our data access tests, we used a fake firestore client that returned hard-coded results. The fake publications that it returned were implemented with Mock.
The firestore client returns an object with an ID property, and a get method that allows us to access the data. We mocked this by specifying that our Mock should have an ID property, and that it should have a get method who's return value is set here. You'll find Mocks are really helpful for emulating the functionality of other objects.
We also used the patch function to swap out the implementation of the internal create app function, which allowed us to verify that the create app function properly sets the data services based on our environment variables. Testing is an aspect of software development that I find difficult at times, especially on existing code bases. However, tests can help us to ensure that the code changes we make aren't breaking the expectations of the app.
In the past, many of the frustrations that I experienced while writing tests, related to the need to patch so many different objects. And this was because classes were instantiating their own external service dependencies. Testing becomes a lot easier if the code that we develop accepts its dependencies as arguments. This allows us to provide alternative implementations without patching.
Early on in the course, we talked about developer quality of life changes. I consider good testing to be a quality of life improvement. Another change we made was to add some Pytest plugins. We used Pytest-sugar to improve the look of the test output. And we used Pytest-cov to check on the code coverage.
I don't consider 100% coverage to be an ideal benchmark for good tests. However, by exercising more of the code, we can improve our chances of finding bugs. For me, the value of a test is more important than the amount of tests. For example, imagine we have a function that accepts an integer, and it performs some calculation, and then it returns the results. It sounds simple, though, what if that integer is negative? Will our code still work? What about a zero value? What if it's a float? What if it's not an integer at all? Somebody passes in a float. Is that code going to do what we expect?
By testing a wide range of values for key functionality, we can have better quality tests. And libraries such as hypothesis can make this sort of testing a bit easier. Another quality of life change that we made was to modify our bash shell using Oh-my-bash. We only configured the theme, which is, in itself, an important improvement. Though we could also include project specific plugins that improve our workflow.
We installed and configured ipdb to replace pdb. And we used the .pdbrc file to create a debugger command alias named interacti, that allowed us to start up an interactive Python shell using IPython. I'm a visual learner. So I find that syntax highlighting, pretty-printing by default, tab completion and magic methods, are all ways of helping me to understand what's happening in the code. And the better I can visualize that, the easier it is for me to debug it in my head when something goes wrong.
All right, so what's next? If, after watching this, the quality of life changes resonated with you, then it might be time to start looking into configuring your development environment. It might be worth reviewing your current development workflow, identifying aspects that you'd like to improve, and then creating a plan to implement those improvements.
If the testing resonated with you, then start looking into some of the different testing frameworks. In this course I used Pytest, 'cause that's what I like. Though it's not the only option. Look into other options, see if they resonate with you.
Once you find an option you like, start reading existing tests that use that framework from open source projects and start writing some of your own. If debugging with an IPython shell resonated with you, then install it, dive into its functionality. It's full of magic commands and other useful tools that make it a bit more user-friendly, at least in my opinion.
Okay, that's going to wrap up this lesson, and with it, the course. I've had a lot of fun creating this course. I hope that you found this helpful. Thank you so very much for watching. And I will see you in another course.
Lectures
Course Introduction - Quality of Life for Developers - What Is It That We're Building? - Exploring the Data Access Layer - The Web Server Gateway Interface - Exploring the Web Application Layer - Exploring the Front End Code - Running the Web App
Ben Lambert is a software engineer and was previously the lead author for DevOps and Microsoft Azure training content at Cloud Academy. His courses and learning paths covered Cloud Ecosystem technologies such as DC/OS, configuration management tools, and containers. As a software engineer, Ben’s experience includes building highly available web and mobile apps. When he’s not building software, he’s hiking, camping, or creating video games.