The course is part of this learning path
Building Solutions for Google Cloud Platform with App Engine
As developers, the learning never ends. Just when we get used to a certain technology, it goes and changes. We’re always needing to learn new languages, frameworks, APIs, and platforms.
And if you’re also in charge of deploying your code, then you need to understand how to setup and configure web servers, deal with scaling issues and manage databases. Frankly, it can be exhausting!
So, why invest the time in learning yet another set of technologies? That’s the question I find myself asking whenever some new trend comes along. I want to know how it’ll make my job, and the jobs of my peers better or easier.
Throughout my career I’ve been responsible for deploying my own code. And I think that’s why the Google Cloud Platform resonates with me so well. It’s a platform that understands developers.
So, why take the time to learn something like App Engine? I think the answer is simple. Because you can take all your development experience and apply it to a platform as a service that removes most of the obstacles to getting code running in production.
The value of having Google ensure your app is highly available is worth a lot to me. We get a rich set of tools for developing locally, and simple deployments, with application versioning. All while using the same programming languages and frameworks that we’re used to.
If you’re looking for a native cloud platform for building out highly scalable applications, or mobile back-ends, then you’ve come to the right place. App Engine provides all that and more.
This course focuses on teaching you about the tools App Engine provides to build out highly scalable systems.
We’ll be building out Python web applications using Flask, and using Cloud Datastore as our database. There is a bit of a learning curve to getting started. And that’s what this course is for, to minimize the amount of time spent learning the platform, so you can get back to writing code.
The source code lives on: Github so feel free to download it, and follow along.
We’ll cover a lot of data in this 2 hour course. And by the end of it, you should feel comfortable getting started building out App Engine applications of your own.
And if you’re looking to get your Qualified Solutions Developer certification, this is going help you with that too.
So, if this sounds useful to you, let’s get started!
In this course
- We’ll create an App Engine application, and deploy it
- We’ll be Developing a REST API with Cloud Endpoints
- We’ll learn about the different Authentication and Authorization methods available on App Engine.
- We’ll learn about the monitoring and management tools available.
- We’ll cover the different storage options available.
- We’ll review Cloud Datastore more in depth.
- We’ll look into ways to improve application performance.
- And We’ll learn about Task Queues.
This is a intermediate level course because it assumes:
- You have development experience
What You'll Learn
|Lecture||What you'll learn|
|Intro||What will be covered in this course|
|Getting Started||Creating our first App Engine application|
|Cloud Endpoints||How to create RESTful APIs with Endpoints|
|Services||How to break our applications down into separate services|
|Authentication||How can we authenticate users?|
|Managing Applications||How do we manage App Engine apps?|
|Storage||How do we use the different storage options?|
|Datastore||A more in depth look at datastore|
|App Performance||How can we make our apps more responsive?|
|Task Queues||How can we run tasks outside of a user request?|
|What's next?||Where do we go from here?|
Welcome back to Developing Solutions for Google Cloud Platform. I'm Ben Lambert, and I'll be your instructor for this lesson. In this lesson, we'll be talking about application management and monitoring. We can't escape it yet. If we're going to build applications, someone is going to be responsible for managing and monitoring them. Luckily, since we're using App Engine, we don't have to reengineer the wheel. Or even work too hard for that matter, since it takes care of providing us with a set of tools needed to make our lives easier.
We've been using the Dashboard throughout the course, though it's mostly just been to click on the link to launch our application. Though you've probably noticed that it offers a lot of functionality. On the Dashboard, we have at-a-glance reporting for metrics related to our app. We can check out these stats for each service and version. These are things such as latency, errors, memory usage and more. And we can adjust the time frame for these charts.
We can check on the individual instances that are running our app, though this becomes more useful when we are dealing with manual scaling. We have the Versions. This is where we can see the different versions of our application, and route traffic to the version we want to. We have Quotas, which allow us to see how we're doing for resource usage.
We have Settings, where we can adjust some basic application-level settings, and even disable the application entirely should we want to. And, under Settings, we can set up a custom domain, which is useful for when we're ready to stop using the Appspot domain and start using our own. And we can also set up SSL certs here too. The overall App Engine UI is full of these useful tools.
Let's dive in a little more in depth into Versions, because this is a feature that is simple to use, however it's very powerful. Up until now, we've developed this app under the same version, V1. Let's make a change to the index.html file. That says V2. And now let's edit the app.yaml file. And we'll make that say V2 as well.
Perfect. Okay, let's push this to App Engine. Once, it's done we'll see that there'll be two versions in the drop-down. Okay, there they are. And if we view the versions, we can see that it's still defaulting to version one. And that's because, by default, if we upload a new version, it doesn't receive any traffic until we say so. However, if we keep uploading new code to the current version the way we have been so far, then the code will continue to go live by default.
Okay, we have two versions. Now what? We have a couple of options. We can use traffic splitting, or traffic migrations. Traffic splitting is a fantastic way to do A-B testing or canary deployments. And the ease of use of the UI makes both of them a trivial thing to do. Check it out. If we click on traffic splitting, we can select the method we want to split by.
There is IP address or cookie. IP address basically creates a numeric cache for each visitor's IP address. And it's going to be a number between zero and 9,999, and it uses this number to determine which version it's going to direct traffic to. And cookie-based will assign a user a cookie with a number between zero and 9,999. And it uses that again to route to a specific version. We're gonna use cookie-based for this demo.
If you want to understand more about how these work, check out the traffic splitting docs. Okay, now we'll add a new version, and we're gonna set it to version two. And now we can adjust with the slider or enter number. And we're gonna set it to 50%. And now, if we browse to the base URL, not one of the versions URLs mind you, just the base URL, we can see it shows version two.
Let's open up the developer tools and delete the cookie. And if we reload, there it is. That's our version one. And if we do this a few more times, it'll vary, based on which one we get. And so, we can use this to slowly add a new version into the mix, and monitor to make sure that there aren't any errors from it. And if everything looks good, we can progressively bump up the percentage, or migrate traffic. Let's see what happens when we set the traffic for version two to be 100%. Notice it's now listed as 100% on the Versions tab, so if we browse to the page, it'll show version two.
Alright, let's migrate back to version one so we can see how migrations work. Notice it's warning us that this is gonna cut traffic right over from version two to version one. It's suggesting maybe we want to do a warm-up migration, and it gives us a link here to check out more information about what that means. Let's see what it has to say. It shows us that, if we add a setting to our app.yaml file for in-bound services set to warm-up, it will ensure the App Engine warms up the instance first by having requests sent to the app which will get everything loaded up and running so that users don't need to wait for that to happen.
Okay, let's go back and migrate our traffic back to version one. And now if we load the base URL, it's version one, great. And if we migrate traffic back to version two, and we check the base URL again, it's version two. So this is a really simple-to-use piece of functionality but it's very powerful.
Next, let's talk about something that lives kind of in-between monitoring and management. And that's resource quotas. There are three types of App Engine quotas or limits. We have the free quotas, the spending limits and the safety limits. App Engine tracks our application's resources against system quotas. It resets all the resource measurements at the beginning of each calendar day, with the exception of stored data, which always represents the amount of data that's currently stored. And logs are included as stored data.
When free apps reach their quota for a given resource, they can't use that resource anymore until that quota is replenished. Paid apps, however, can exceed the free quota until their spending limit is exhausted. And safety limits can never be exceeded. They're in place to ensure that App Engine remains highly available for everyone. Daily quotas get replenished daily at midnight Pacific time, and per-minute quotas are refreshed every 60 seconds.
We've mentioned the spending limit a few times. Let's check out how to set it, because by default, there isn't a spending limit. We can set the daily spending limit in the spending field on the Settings page. This will help to ensure that we're not gonna go broke should our application go viral overnight. The thing to remember about quotas is once we hit our free cap, that's it. We can't use that resource anymore, unless we upgrade to a paid account or the resource quota is replenished. And paid account quotas will allow us to go beyond the limits of the free quota. We'll only run into an issue with paid accounts if we hit the spending limit or the safety limit.
Okay, let's shift our attention to logging. We checked out the Dashboard earlier, and it's great for at-a-glance info about our app. Though, sometimes we run into an issue where we need to parse the request or application's logs for more details. Let's check out the logging that's peppered throughout the app. By default we're using the Python logging API. And we can see that it's just logging some comments here and there as info before certain actions happen.
Alright, let's jump into the console and see what it looks like. If we click on the menu, we'll see the logging option. And we're greeted with the recently logged info. So right off we can see, our application's requests are logged. This is done by default from App Engine logging and we don't need to do anything to enable this. And if we expand this /me request here, we can see that it says current user, email@example.com and that's because we added that line to our Python code with the logger. So we can see that our application's logs for a given request are here. And we can filter the logs.
Now we have a lot of options for filtering. We're gonna use the text filter, and we're gonna search for the word render. And there it is. We have some results coming from our homepage. And if we clear out our filters and set logging level to error, we have a list of all of the requests that have returned either a Non 200 level status, or used the error-logging handler. And this makes it easy to find specific types of log messages.
Now sometimes, we'll need to do something else with the logs. Maybe we need to process them with something that we can customize a bit more. So App Engine allows us to export the logs quite easily. And we can export them to things like BigQuery, Cloud Storage, or Pub/Sub. And this will allow us to either store back-ups, or process the logs, maybe for like security audits or something of that nature.
So if the built-in logging tools aren't enough, we can use the exports to customize things. Now, along the same lines as logging, there's another tool that will help us to monitor things. And that's called the trace tool. It'll allow us to see the latency for our requests. This is really useful to help us keep tabs on things and make sure that we don't introduce any changes that negatively impact the app's performance. We can drill into common routes, and even filter the data. This helps us to find areas of our application that may be slow, and providing poor user experience.
Alright, let's summarize what we've covered so far. App Engine provides us with tools for managing the deployment of versions and services, including traffic splitting, to incrementally deploy or to perform A-B testing. And, we can use traffic migration to switch traffic from one version to another. When we're on the free plan, and by that I mean we haven't enabled billing, we have a set of free-tier quota limits. If we hit those limits for a resource, we can't use that resource again until the quota is reset, though we can increase those limits by upgrading to a paid plan. Last but not least, App Engine allows us to use the log viewer to see request and application logs.
Alright, the demo application we used for this lesson uses a couple storage options we haven't talked about in this course just yet. So, let's do that in the next lesson. Let's review the storage options available to App Engine. So, if you're ready, we're gonna learn more about cloud storage, cloud sequel, and cloud data store.
Alright, let's dive in.
About the Author
Ben Lambert is the Director of Engineering and was previously the lead author for DevOps and Microsoft Azure training content at Cloud Academy. His courses and learning paths covered Cloud Ecosystem technologies such as DC/OS, configuration management tools, and containers. As a software engineer, Ben’s experience includes building highly available web and mobile apps.
When he’s not building the first platform to run and measure enterprise transformation initiatives at Cloud Academy, he’s hiking, camping, or creating video games.