The course is part of this learning path
Building Solutions for Google Cloud Platform with App Engine
As developers, the learning never ends. Just when we get used to a certain technology, it goes and changes. We’re always needing to learn new languages, frameworks, APIs, and platforms.
And if you’re also in charge of deploying your code, then you need to understand how to setup and configure web servers, deal with scaling issues and manage databases. Frankly, it can be exhausting!
So, why invest the time in learning yet another set of technologies? That’s the question I find myself asking whenever some new trend comes along. I want to know how it’ll make my job, and the jobs of my peers better or easier.
Throughout my career I’ve been responsible for deploying my own code. And I think that’s why the Google Cloud Platform resonates with me so well. It’s a platform that understands developers.
So, why take the time to learn something like App Engine? I think the answer is simple. Because you can take all your development experience and apply it to a platform as a service that removes most of the obstacles to getting code running in production.
The value of having Google ensure your app is highly available is worth a lot to me. We get a rich set of tools for developing locally, and simple deployments, with application versioning. All while using the same programming languages and frameworks that we’re used to.
If you’re looking for a native cloud platform for building out highly scalable applications, or mobile back-ends, then you’ve come to the right place. App Engine provides all that and more.
This course focuses on teaching you about the tools App Engine provides to build out highly scalable systems.
We’ll be building out Python web applications using Flask, and using Cloud Datastore as our database. There is a bit of a learning curve to getting started. And that’s what this course is for, to minimize the amount of time spent learning the platform, so you can get back to writing code.
The source code lives on: Github so feel free to download it, and follow along.
We’ll cover a lot of data in this 2 hour course. And by the end of it, you should feel comfortable getting started building out App Engine applications of your own.
And if you’re looking to get your Qualified Solutions Developer certification, this is going help you with that too.
So, if this sounds useful to you, let’s get started!
In this course
- We’ll create an App Engine application, and deploy it
- We’ll be Developing a REST API with Cloud Endpoints
- We’ll learn about the different Authentication and Authorization methods available on App Engine.
- We’ll learn about the monitoring and management tools available.
- We’ll cover the different storage options available.
- We’ll review Cloud Datastore more in depth.
- We’ll look into ways to improve application performance.
- And We’ll learn about Task Queues.
This is a intermediate level course because it assumes:
- You have development experience
What You'll Learn
|Lecture||What you'll learn|
|Intro||What will be covered in this course|
|Getting Started||Creating our first App Engine application|
|Cloud Endpoints||How to create RESTful APIs with Endpoints|
|Services||How to break our applications down into separate services|
|Authentication||How can we authenticate users?|
|Managing Applications||How do we manage App Engine apps?|
|Storage||How do we use the different storage options?|
|Datastore||A more in depth look at datastore|
|App Performance||How can we make our apps more responsive?|
|Task Queues||How can we run tasks outside of a user request?|
|What's next?||Where do we go from here?|
Welcome back to developing solutions for Google Cloud Platform. I'm Ben Lambert, and I will be your instructor for this lesson. In this lesson, we'll talk about Services. We're gonna talk about what they are and how we can use them. App Engine applications at their core are made up of one or more services. Services allows us to break down apps into different components.
So, maybe we will have our core application which is technically the default service. And we can have a separate service that serves up an admin dashboard. And services are technically independent apps so you can use different runtimes for each if you need to. This can be useful in a lot of scenarios, let's say our core application is built with the Python runtime and then another team is building up some functionality in Go and we can have the Go app as a service and have it run under our App Engine app. And this will allow us to share the same state through Datastore and Memcache and they can interact with each other through Task Queues as well.
Okay, let's build our own service. We're gonna do something simple, we're gonna make a service in Go that just says "Hello" to us. We're gonna start by adding a folder. The common structure for services is to have them place in their own folder and it will contain a source code and a yaml config files that will look like our app.YAML file. Okay, now let's add a file called greeter.go And let's have some coding here. We'll set the package name and add a couple of import statements. Now, wanna create an init function, and this will pickup the mapping of our route to our actual function. And so we will map the base URL to a function we'll call handler. And so we're gonna have to create that handler function. And this is the function that will be run anytime someone visits that base URL. And we will just have it return this nice greeting here, "Hello, from Go".
Okay, so now we have some code and since services are really their own apps, we're gonna need a configuration file, Go like Python uses yaml and the convention is to name the config file after the service, so we are gonna call this greeter.yaml and we're gonna specify the application name and the service name of greeter and then the version will be v1, the runtime is gonna be Go and the version is gonna be Go1 And now we are gonna setup some URL handlers, we just need the one for that base route and then we're gonna add the script of underscore go underscore app and this is just for the development server, it’s gonna get ignored in production. Now, let's deploy this and see what happens.
We've used the app launcher up to this point. However, app launcher doesn't know about our new service. So for this, we're gonna use the command line to deploy our service, we're gonna run appconfig.py, update, greeter/greeter.yaml This looks in our configuration file and uses the info so it can understand how to deploy it. So, if we run this we get an error. So, what happened and why? The error is module parameter not specified. Now, what is the module in this context? Services used to be called modules, so on occasion you're gonna see something in the tools or the documentations or blog posts that reference modules and not services. Just so you might wanna keep in mind modules and services are the same thing in the context of app engine. So, we'll just change this in the config file to make the same module and if you run the command again, it’s gonna take a moment to deploy our new service. Though, once it does we'll be able to test out in app engine.
Okay, there it is. Notice that we have a new dropdown, when we only had the default service before, we don't need it. But now that we have multiple services, we can view each service on its own. And if we set it to greeter, and then we click this URL on the right, it will take us to that service so we can see our greeting from our Go code and if we go back to the default service, we see our original message. Notice the URL for the default service, its our project ID.appspot.com. And then look at the URL for our greeter service, you notice it’s different. It breaks out our service by its name. The way app engine handles routing gives us a lot of flexibility.
Let's put our demo on hold for a moment and talk about how routing works with app engine. When we use app engine, we use one or more services and each service can have one or more versions. So, app engine allows us to target these different services and versions and in some cases, even the VM instances. If we are using the appspot.com domain, then this is how URLs are broken down. We have instance dash dot dash dot version dash dot dash service dash dot dash app and then the appspot dot com domain. Notice that the word dot is spelled out, and there's a hyphen on either side of it. That's because google doesn't issue a cell search with double wild card domains for appspot. So for appspot we can target specific versions and services. It starts with the VM instance then the version, the service and then the app ID. So we can progressively drop the leftmost part of the URL and get less specific versions.
So if we remove instance and we won't target the specific VM if we remove version, we will get the current version. If we remove service, we're gonna get the default service. And if we remove app then we're gonna be redirected back to our console because app engine kind of thinks we've lost our mind. So, if we want to use our own custom domain, then we can do that and we don't need to use that special dash dot dash syntax. Instead we can just use a period. Since most of us we are using our own custom domain, this is the way they will access different services and versions. This talk of URL is fastened to our demo because we created a new service and we deployed it however it is currently only access through the URL pattern we just talked about. You can view it at greeter.ca-gcp-developer.appspot.com. What if we wanted it to be accessed via a different URL path? For example, what if we wanted to use the same base URL and then access it through /greeter? App engine allows us to do this through the use of a configuration file called the dispatch file. It’s a yaml file for basically all runtimes except for JAVA words and XML file. This is basically a rule across the board, JAVA uses XML configuration files and everything else that's going to use yaml.
So let's see how this works, let's go back to our text center. And we are gonna create a dispatch.yaml file. We're gonna add in an application which is our project ID. And we'll add a dispatch section and this is where we can map URL patterns to different modules. So we'll add a URL pattern here /greeter. And anything that has this URL and it will get directed to the greeter module. Remember that modules are synonymous with service in this context. Okay, let's deploy this. Now, we're gonna use a command that we haven't used yet. It’s the appconfig update dispatch command. And it expects us to fasten the folder it should look into the dispatch file. And it’s the current directories, we're just gonna use a period and it runs pretty quickly.
So if we go back to the console, we can test this out. So now if we use the default services URL, we can see that it returns the message we expect. And now if we add the /greeter at the end of it, it displays our message from our go app. So we have some ability to override the routing behavior. As of the time of this reporting, we have a limit of 10 dispatch URLs. Okay you may have noticed, we wrote some code and deployed it to app engine without running it locally. I wanted to loop back to the development environment server once we talk about services just a bit more because our local development server is where we run things a bit differently when it comes to multi-service apps. App engine treats each service as its own running app. So when we start up with app launcher, it doesn't know that we want to start other services. Now this is a good thing because we might not want to start each service every time we start up our core app.
So if we want things to start up with app launcher, we need to tell it where to find these other services. And we can do this by adding a command on flag and it will point to our greeter.yaml file in this case. Now if we start it up, it will run both. When I work with multiple services, I like to work on the command line. The app launcher works just fine for this stuff, this is just my personal preference because as our apps get more complicated, it allows me to use the interactive debugging with a bit more ease. So if we're gonna run this ourselves on the command line, we're gonna use the dev app server. And then we're gonna list off the config files that we want to use, we'll add our app.yaml and our dispatch and our greeter.yaml. Okay, when we run like this, we're gonna see some URLs listed off here and they represent the different service URLs.
Now we also have this dispatch URL, and that gives us the central URL for routing. So if we open up linkup, its gonna show us the default service. And if we add the URL /greeter, it going to use the dispatch file and its gonna redirect us to our Go service. Alright, we can override the default routing with the dispatch file and we can test the different services with the launcher or the dev server. But how do we actually interact with our different services? We could use some of the helper methods in the app engine API to help us build the URL for the other services and interact with them via http. This method here in Go, calls the app engine "module hostname function" and we can use it to get a URL for our default service and then we can target a specific version of it. Using this makes it easy to build URLs that allows to interact with the other services. Though you may recall that we talked about shared state before.
We can also use cloud datastore and memcache as a mechanism for shared state. So we aren't limited to interacting with our other services strictly through http. If we change an entity and datastore in one service and then save it, we're gonna see that the same change when we fetch it from another service. Let's check that out. I've pre-written some code to make it easier. If we look at this Python file, we have an import statement for the ndb library which is an ORM-like obstruction layer and we have a greeting class which represents the datastore entity and has a single property called "message". And then we have this route here and it takes a message that we passed in through the URL and it saves it as a new message to datastore. We're gonna cover datastore later on, so don't worry if it feels like we're skipping ahead of it.
Okay, let's test this out. We're gonna hit that URL and we'll say "hi". And if we use the admin dashboard, we can see under the data viewer that we can list off our datastore objects. If we look at the greeting entity, we see one object and its our "hi" message. Okay, let's add another. And it will say "hello", and if we refresh the viewer, right there it is. Let's add one more and we'll say “hola". Alright, now let's look at our Go code. We've implemented the datastore library, we've created the structs to represent the greeting and we've changed the code in our base URL to fetch all greetings and add from Go to the end of the message. Okay if we browse to this URL, we'll see that it lists off the original messages, albeit without any spaces. And now if we go back to the admin dashboard and look at the datastore viewer, we can see that it updated the entities that we created with Python. And this works with memcache as well.
So we can use these to share state between our services and that gives us a lot of power. So when it comes to using memcache for shared state, there's a few things we should keep in mind. First, it is very fast. Which makes it a great option however, we only want to use it for data that we can easily recreate. Memcache can, for a few reasons drop records. Things such as memory pressure can cause old records to get booted to free up room for new ones. There are other issues that could cause record skid drop before their intended expiration though they're pretty low level and rarely occur. Just keep in mind, it is possible that when we attempt to fetch records from memcache, the records might not be there. Don't let this worry you when you use it for what it is intended for this really isn't going to be an issue. And we'll talk about that in another lesson. For now, just remember if you absolutely need to ensure that the data is there, and that its persistent, you want to use something like datastore. Now we can also use Task Queues to send work between services and we'll cover that later too.
Alright, let's summarize what we've covered. Services, which used to be called modules. Are a way to break up our applications into separate apps. Every App Engine app has at least one service named default and services can be versioned independently of each other. Typically, we'll find services are separated into their own folders in our application directory structure. Each service can have its own runtime so we can mix runtimes as we need to. And we can share application state using Memcache and Datastore and we can also use Task Queues to go across service for sharing work. When it comes to routing, our App Engine app, the URL pattern is instance, version, service and app. And if we're using the appspot domain, and those were separated by a dash, the word dot and then another dash. So, if we have an app called "pizza time", and a service named "orders", we could access it through the URL, orders-dash-dot-dash-pizza-time.appspot.com And if we try and hit a URL where one of those things doesn't exist, its gonna take us to the default service for that app.
Now we've created an app and we've created some services, however, we haven't locked anything down. How do we keep our applications and different sections of our applications from prying eyes? This is going to be the topic of our next lesson when we talk about authentication. So if you're ready, let's get started.
About the Author
Ben Lambert is the Director of Engineering and was previously the lead author for DevOps and Microsoft Azure training content at Cloud Academy. His courses and learning paths covered Cloud Ecosystem technologies such as DC/OS, configuration management tools, and containers. As a software engineer, Ben’s experience includes building highly available web and mobile apps.
When he’s not building the first platform to run and measure enterprise transformation initiatives at Cloud Academy, he’s hiking, camping, or creating video games.