The course is part of this learning path
Building Solutions for Google Cloud Platform with App Engine
As developers, the learning never ends. Just when we get used to a certain technology, it goes and changes. We’re always needing to learn new languages, frameworks, APIs, and platforms.
And if you’re also in charge of deploying your code, then you need to understand how to setup and configure web servers, deal with scaling issues and manage databases. Frankly, it can be exhausting!
So, why invest the time in learning yet another set of technologies? That’s the question I find myself asking whenever some new trend comes along. I want to know how it’ll make my job, and the jobs of my peers better or easier.
Throughout my career I’ve been responsible for deploying my own code. And I think that’s why the Google Cloud Platform resonates with me so well. It’s a platform that understands developers.
So, why take the time to learn something like App Engine? I think the answer is simple. Because you can take all your development experience and apply it to a platform as a service that removes most of the obstacles to getting code running in production.
The value of having Google ensure your app is highly available is worth a lot to me. We get a rich set of tools for developing locally, and simple deployments, with application versioning. All while using the same programming languages and frameworks that we’re used to.
If you’re looking for a native cloud platform for building out highly scalable applications, or mobile back-ends, then you’ve come to the right place. App Engine provides all that and more.
This course focuses on teaching you about the tools App Engine provides to build out highly scalable systems.
We’ll be building out Python web applications using Flask, and using Cloud Datastore as our database. There is a bit of a learning curve to getting started. And that’s what this course is for, to minimize the amount of time spent learning the platform, so you can get back to writing code.
The source code lives on: Github so feel free to download it, and follow along.
We’ll cover a lot of data in this 2 hour course. And by the end of it, you should feel comfortable getting started building out App Engine applications of your own.
And if you’re looking to get your Qualified Solutions Developer certification, this is going help you with that too.
So, if this sounds useful to you, let’s get started!
In this course
- We’ll create an App Engine application, and deploy it
- We’ll be Developing a REST API with Cloud Endpoints
- We’ll learn about the different Authentication and Authorization methods available on App Engine.
- We’ll learn about the monitoring and management tools available.
- We’ll cover the different storage options available.
- We’ll review Cloud Datastore more in depth.
- We’ll look into ways to improve application performance.
- And We’ll learn about Task Queues.
This is a intermediate level course because it assumes:
- You have development experience
What You'll Learn
|Lecture||What you'll learn|
|Intro||What will be covered in this course|
|Getting Started||Creating our first App Engine application|
|Cloud Endpoints||How to create RESTful APIs with Endpoints|
|Services||How to break our applications down into separate services|
|Authentication||How can we authenticate users?|
|Managing Applications||How do we manage App Engine apps?|
|Storage||How do we use the different storage options?|
|Datastore||A more in depth look at datastore|
|App Performance||How can we make our apps more responsive?|
|Task Queues||How can we run tasks outside of a user request?|
|What's next?||Where do we go from here?|
Welcome back to Developing Solutions for Google Cloud Platform. I'm Ben Lambert, and I'll be your instructor for this lesson. In this lesson, we'll talk about authentication and authorization, since they're typically a big part of applications.
Let's start with a basic definition of the two. Authentication verifies the identity of a user, and basically, that just means it confirms they are who they say they are. And authorization is about determining the permissions someone or something has to a resource. The reason we say someone or something is because users could be authorized to access certain areas of our applications. However, we could also have a user authorize us to access their data from Google services. And we could be authorized by an entity such as Google to access APIs used in our apps. So, authorization is not a one-way street.
When it comes to authentication on App Engine, that is to say, verifying a user's identity, we have a few options. We can use the Firebase authentication. It's an option that allows us to use federated authentication, supporting Google, Facebook, Twitter, and others, to give us the widest user base, and relative to the amount of support for those different options, it has the fewest lines of code to implement.
We could also use the Google Sign-in, which would allow us to put a log in button on our app and allow users with a valid Google account to log in, and that's a very easy option to use. We can also use the OAuth2 and OpenID, which allows us a lot of customization.
However, we're gonna focus on the Users API to start with, because it's integrated into App Engine really well, and it allows us to very easily get started with authentication for users.
All right. A Users API allows us to have any user with a valid Google account or Google Apps account to authenticate. And with this, we'll be able to use basic role authentication. We've talked about roles in the fundamentals course. Owner, Editor, and Viewer. By default, App Engine apps don't require any sort of authentication.
So, how would we go about ensuring that users are authenticated with the Users API? All right, for this, we have two options. We can use configuration settings, or we can do it programmatically. The configuration option allows us to push the config files without changing the code, so it's really cool. There's a lot of value to just changing it in the config file, and then uploading that. And if we need to, we can also do it in code.
Now, we've gotten to this point where we're building a simple Hello, World! app for each of our demos, but it's not really gonna work from here on out. So, from here out, we'll be using a demo application that's created specifically for this course, called 1000 Words. As in, "a picture is worth a thousand words." And it allows us to have users upload an image, set its category, and then add a message. And then those images are displayed for everyone to enjoy. So, let's take a look at the app before we start locking it down.
Here's the app. It's just a couple of pages. We have our home page that displays the recent image for each category, and if we click into the category, we'll see the images for the category, and if we click on one of the images, it drills into the individual image. These images are stored on Cloud Storage, and we're using Cloud data storage to wrap the categories and images, so that we can do some basic querying on the data, and we're gonna expand more up on that later in the course. And we have a profile page, and that allows us to log in and out. And when we're logged in, it shows us a welcome message and a log out link.
It's a fairly simple UI, so let's check out the source code in the config files. If we go into the app.yaml, we have several different URL handlers. We have this one here at the top that handles some static files, and it has this handy application readable setting, and that allows us to read these files from disk from inside of our application. This here is just used for initializing the app with some images, so we'd be able to remove that in production.
Let's test it out. We'll reload the page. And notice that we have this log in box. Okay, it's kind of an ugly screen, however, this is just the development server log in. Its job is to fake the functionality of the real log in page, and we'd see if this was actually deployed. Let's enter an email address. And then click log in. Notice it redirects us back to the profile page, and it's welcoming us and displaying our email address. And it has this log out URL.
Let's log out, and we're gonna log in as a different user. The default is this test user. Notice that there's no password here. It's a fake log in for development, so it really doesn't matter what we enter here. Did you notice that there is this is administered checkbox on the page? It allows us to set a fake user as an admin. What if we wanted to have a page that's only accessible to admins? For that, we just change the log-in setting from required to admin, and then, if we reload the page, notice it doesn't allow us to view the page. It tells us that we need to be an admin. So, let's log in again and pretend to be an admin, and there. We can now view the page. Let's change it back to required from admin, so that we don't need the user to actually be an admin just to use this site.
All right. For configuration, it's really that simple just to lock down a script. By default, there's no off. However, we can set required or admin and allow App Engine to handle the rest. All right, let's check out the code that populates the nickname and the log out URL. If we look in this template here, it says if the user variable is set, then display the welcome and a log out URL. And if we look at the code that populates this template context, we can see that we're importing the Users API, and down here in the route, we're calling the get current user and create logout url method, and we're adding it to the context dictionary and then passing that into our template.
So, if our application has simple requirements, we can use the Users API. It'll allow us to authenticate users, and with a simplistic admin check, we can do a form of authorization. Any user that we've added to our IM users, with either a role of Owner, Editor, or Viewer, is considered an admin in the context of the Users API. With the Users API, we can access the user's email address, a unique ID for them, and their nickname. So, this makes it easy to use this method of authentication with limited admin-based authorization.
All right. Sometimes, we need more data from our end users, or maybe our app is going to be managing data on their behalf. For that, we can use the OAuth2 client. I've already installed it here with PIV, so let's check out how to use it. Let's say that we want to be able to list off a bunch of files from Google Drive. In our case, we're gonna say that it's because we eventually want to allow users to upload an image that they have in their drive. For this, we'll need to prompt the user to give us authorization to use their info.
Let's look at the end result, and work our way back from there. All right, first, you can see that we have this link on the side nav for uploading files from Drive, and if we click that, we have another link to list files from Drive, and so we click that, and we get a listing of files from my Google Drive. Let's go through this and see how it's actually done. It starts by heading into the console and clicking on the menu, and selecting the API Manager. And we click the Credentials link on the side. And then we click on the OAuth Consent Screen in the middle of the page. This is the screen that will show to our users when they're first prompted to allow us access to their data. I've already filled out this info. You can see the product name and the home page URL are set.
Okay, let's click on the Credentials tab. And here, we have three different types of credentials that I've pre-generated. We have a simple key, which allows us to authenticate our application to Google's APIs. This allows us to use any of the APIs that don't interact with the user's data. So, things such as the URL Shortener or the Translate API.
And then we have the OAuth2 client IDs. These are the keys that allow us to request access to a user's data. We'll be using this key to access the data from our user's Google Drive. And then, we have the service account keys, and this allows us to have our code run under a specific service account. We'll see this later on, when we interact with the Vision API. So, to access a user's data, we're gonna download this OAuth2 key and save it to our project. This file needs to be kept secret. That means don't add it to your source code repos, and make sure it isn't someplace that it can be publicly downloaded, such as a static folder. Okay. We'll save it to our project directory and rename it to client_secrets.json.
All right. Now, we need some code to handle this OAuth stuff. Let's check out the code that handles the route for the profile. It's in our fromdrive.py, and starting at the top, it looks a lot like the other files so far. Scrolling down, we have two new imports. We have the OAuth2 client, and the API client modules, and if we keep scrolling down, we have the route that handles the from drive get request. All that does is grab the template and return it. There's really nothing interesting there.
So, let's skip down to the route at fromdrive/oauth2callback. Looking through this code, here's what's happening. We're starting the process with the flow from client secrets method. This is where we specify that JSON credentials file that we downloaded earlier. We also specify the scope, which is the resources and data that we want to access from our users. When doing this, we want to use the Least Privileged Principle when requesting access to a user's data. As a rule, don't ever take more than you need. You can always prompt them for more specific permissions later on.
Okay. Now, we're looking in the URL param, for a parameter named Code. And if it's there, we kick off the next step in the process. Otherwise, we try the process again. If it is there, we use it to get the credentials with the step two exchange method, then we store the credentials in session state and redirect back to the original route.
So, this OAuth2 callback method will allow us to prompt the user, and if they approve of letting us access their info, we can start using the APIs to interact with their data. Let's check out the from drive files method. It's responsible for listing off the files from Google Drive. It starts out by making sure that we have the credentials. Otherwise, it kicks off the off process. So if we have them, we use the from JSON method to grab the credentials saved in session state, and create a credentials object. If the credentials are expired, again, we kick off the off process, then we use the authorize method to get an HTTP object that will be set up to include the headers needed to make requests from the APIs we've been granted authorization for.
Now, we can use the build method, and it's gonna allow us to perform service look-ups, and it acts as a wrapper around the APIs we want to use. So we can specify that we wanna use the Drive API, and Version 3, in particular, and then, we use the object return to run the files call and then list them off. And then, we wrap the info that's returned in the context, and we pass it off to the HTML template. It's worth mentioning there are helper utilities, such as decorators that will remove a lot of this boilerplate code. However, I wanted to show this way, so that when we use these helper methods, we know what the code is doing for us.
Okay. Let's check out the template. You can see that there's really not much to it. It looks to see if we have any files, and if so, it loops over them and displays them. When we're developing, it's often useful to have some mechanism to test out the different API calls, and for this, we can use the OAuth Playground. This is the tool that'll help see the structure of the data return. I find it really helpful to be able to test this stuff out and see what the data that needs to be sent looks like, and what it will get back.
Okay. Let's check out how to use a simple key to access an API that doesn't require user's data. At the bottom of our individual image page, we have this Get Short URL link, and if we click it, we get a shortened URL back. Let's look at the code behind this. It's in the images.py file, in the short_url function. It's using the build method again, and we're passing in the developer key param, which is the string that we were given when we clicked on that link for the API key.
Again, we're gonna wanna make sure that this is kept secret, because if other people use it, it could rack up quite the bill. So, we can use simple API keys to interact with the different Google APIs. Now, what if we want to interact with something like the Vision API, and have it access data from our Cloud storage buckets? For this, we can use service accounts. Service accounts allow us to run code under a specific account, so we'll be able to allow the Vision API access to things such as the objects in our storage buckets.
Let's check it out. We have a service that's responsible for running tasks, and we'll get into tasks later on in the course, but for now, we're just gonna focus on the parseimage.py file. In the get tags from image function, we run the get application default method to fetch the default credentials. This is the default service account for App Engine apps. And then, we use the build method to locate the Vision API and build a wrapper around it, passing in the service account credentials. Then, we build the object that will pass the image the user uploaded off to the Vision API and get the labels back, and we'll use these labels as our tags for the image. So, running this under the service account allows the API to have access to our resources, such as storage.
All right. Let's summarize what we've covered so far. We have a few options when it comes to authentication, and the Users API is the easiest to use and set up. It's limited in its ability, though it allows us to authenticate users with little or no code.
When we use the configuration option to protect our routes, we can use either Optional, Required, or Admin. Admin in this context means any user that is either an Owner, Editor, or Viewer role for the project.
And when we need to get more info from our end users, or we want to interact with their data, we can use OAuth to allow them to authorize us access to their info. All right.
At this point, we've been deploying apps to App Engine quite a bit, so it's about time that we talked about how to manage our apps, as well as how to monitor them. And that's gonna be the topic of our next lesson.
So, if you're ready to dive into application management, let's get started.
About the Author
Ben Lambert is the Director of Engineering and was previously the lead author for DevOps and Microsoft Azure training content at Cloud Academy. His courses and learning paths covered Cloud Ecosystem technologies such as DC/OS, configuration management tools, and containers. As a software engineer, Ben’s experience includes building highly available web and mobile apps.
When he’s not building the first platform to run and measure enterprise transformation initiatives at Cloud Academy, he’s hiking, camping, or creating video games.