1. Home
  2. Training Library
  3. Google Cloud Platform
  4. Courses
  5. Developing Solutions for Google Cloud Platform with App Engine

Storage

Contents

keyboard_tab
Introduction
1
Course Intro
PREVIEW2m 31s
2
Getting Started
PREVIEW12m 23s
App Engine
4
Services
15m 53s
7
Storage
11m 19s
8
Datastore
9m 28s
Wrap Up
11
Summary
8m 35s

The course is part of this learning path

Google Cloud Platform for Developers
course-steps 2 certification 1 lab-steps 2 quiz-steps 2
play-arrow
Start course
Overview
DifficultyIntermediate
Duration2h 1m
Students860
Ratings
4.8/5
star star star star star-half

Description

Building Solutions for Google Cloud Platform with App Engine

As developers, the learning never ends. Just when we get used to a certain technology, it goes and changes. We’re always needing to learn new languages, frameworks, APIs, and platforms.

And if you’re also in charge of deploying your code, then you need to understand how to setup and configure web servers, deal with scaling issues and manage databases. Frankly, it can be exhausting!

So, why invest the time in learning yet another set of technologies? That’s the question I find myself asking whenever some new trend comes along. I want to know how it’ll make my job, and the jobs of my peers better or easier.

Throughout my career I’ve been responsible for deploying my own code. And I think that’s why the Google Cloud Platform resonates with me so well. It’s a platform that understands developers.

So, why take the time to learn something like App Engine? I think the answer is simple. Because you can take all your development experience and apply it to a platform as a service that removes most of the obstacles to getting code running in production.

The value of having Google ensure your app is highly available is worth a lot to me. We get a rich set of tools for developing locally, and simple deployments, with application versioning. All while using the same programming languages and frameworks that we’re used to.

If you’re looking for a native cloud platform for building out highly scalable applications, or mobile back-ends, then you’ve come to the right place. App Engine provides all that and more.

This course focuses on teaching you about the tools App Engine provides to build out highly scalable systems.

We’ll be building out Python web applications using Flask, and using Cloud Datastore as our database. There is a bit of a learning curve to getting started. And that’s what this course is for, to minimize the amount of time spent learning the platform, so you can get back to writing code.

The source code lives on: Github so feel free to download it, and follow along.

We’ll cover a lot of data in this 2 hour course. And by the end of it, you should feel comfortable getting started building out App Engine applications of your own.

And if you’re looking to get your Qualified Solutions Developer certification, this is going help you with that too.

So, if this sounds useful to you, let’s get started!

Course Objectives

In this course

  • We’ll create an App Engine application, and deploy it
  • We’ll be Developing a REST API with Cloud Endpoints
  • We’ll learn about the different Authentication and Authorization methods available on App Engine.
  • We’ll learn about the monitoring and management tools available.
  • We’ll cover the different storage options available.
  • We’ll review Cloud Datastore more in depth.
  • We’ll look into ways to improve application performance.
  • And We’ll learn about Task Queues.

Intended Audience

This is a intermediate level course because it assumes:

  • You have development experience

What You'll Learn

Lecture What you'll learn
Intro What will be covered in this course
Getting Started Creating our first App Engine application
Cloud Endpoints How to create RESTful APIs with Endpoints
Services How to break our applications down into separate services
Authentication How can we authenticate users?
Managing Applications How do we manage App Engine apps?
Storage How do we use the different storage options?
Datastore A more in depth look at datastore
App Performance How can we make our apps more responsive?
Task Queues How can we run tasks outside of a user request?
What's next? Where do we go from here?

 

Transcript

Welcome back to developing solutions for Google Cloud Platform. I'm Ben Lambert and I'll be your instructor for this lesson. In this lesson, we'll talk about the different storage options available to us. We'll talk about Cloud storage, Cloud SQL, and Cloud Datastore. We discussed these in the fundamentals course just a bit. Cloud Storage is for blob storage, it's useful for videos, images, html, JavaScript, and CSS files. Cloud SQL is a managed version of MySQL that allows us to use a relational database without needing to worry about managing and scaling databases. This is great for existing web frameworks.

Things such as Django or Rails on flexible environments, or one of the many PHP frameworks. And its also great if we have relational data and we need to be able to run complex queries. And Cloud Datastore is another queryable storage option; which is highly scalable. However it doesn't support complex queries. Let's go into each of these a bit more. Cloud Storage is great for storing files; however it's not a file system. It's a highly scalable object storage option. We can access it through the REST API, through the client libraries or the gsutil command.

Let's check out how to actually use buckets and objects inside of code. We'll use this same thousand words application we've been working with. It uses Cloud Storage to store the uploaded images and then it displays them on the webpage using their public URL. And this ensure that our app doesn't need to handle serving up the files.

Let's look at the code in the upload .py file. In the create image method, We handle the users form submission. The part that's relevant to talking about Cloud Storage is just a ways down. We take the uploaded file and we pass it into a function called save uploaded file, along with a folder name. Let's check out the code for that function. It lives in the storage .py file. We start out with some imports. We'll be using the Cloud Storage module. We use the app identity API to fetch the default bucket name. And then we have these helper methods at the top here, that determine the appropriate base URL to use.

For local development it will be a bit different than the actual Cloud Storage URL. And then we fetch a bucket name from our environment variable if it exists, or the default applications bucket. And then we have this write method, and it accepts a file name, the content type, and the content to write. And then optionally a flag to control if the file is public or not. And I'm opting to default it to public for this application. If we check out this line here, we can see that we're using the open method to open up the file on Cloud Storage, so we can write to it. The same way we would if it was just a local file. And then once we have an open object we write the data and we close it up and then we're done. This should feel pretty familiar to us as developers.

And then we have this read method; which, again, opens the file, it reads the contents, and then closes it up before returning the content. And then here we have this helper method. And this is the method we're using in our form submission. It accepts an uploaded file, it handles the writing, and then returns the file name so we can save it in our Datastore object. The Cloud Storage module makes it really easy to open up, read, and write files. We can also do things like delete files, copy them, merge them, and other cool things like that. And, if we don't need to handle this stuff in code we can use the gsutil command, or the console. So we're not limited.

Okay. Next up lets talk about Cloud SQL. This is a Google managed MySQL database. It allows us to scale vertically very easily for reads and writes. And we can scale horizontally for reads. The great part about Cloud SQL is, that if we've ever used MySQL in the past, or even any relational database, then the learning curve is practically nonexistent. Now, if we're gonna use Cloud SQL with App Engine we want to make sure that it's located in the same region as App Engine. And then Cloud SQL will automatically follow our App Engine app to different zones as needed, to ensure low latency. And that's really cool.

We can connect to Cloud SQL in code via the MySQLdb module in python, and with JDBC, JPA, or JDO for Java. When we develop locally, using local MySQL instances often make life easier. So, typically we need a way to determine if our code is running locally or if it's an App Engine. And for that we can check the server software environment variable. If it starts with Google App Engine, then we are running in, you guessed it, App Engine. And if it not we can assume that we're local or at least outside of the context of App Engine. Okay. Because Cloud SQL is just an existing MySQL database, there's really nothing new to show here. So we'll wrap up talking about Cloud SQL and we're gonna move on to Cloud Datastore.

Datastore is a NoSQL database that's highly scalable. It's designed to allow both strong and eventual consistency. And that allows us a lot of flexibility because it offers us a wider variety of use cases. Let's cover some of the terminology. We have kinds, entities, and properties. A kind is similar to a table in SQL terminology. And entity is like a row of data in SQL. And a property is like a column. With Datastore, we use unique keys to identify entities. We can use a string of our own data or an automatically generated integer.

We can even group entities together to form entity groups; which allows for strong consistency inside transactions. The entities inside of groups don't need to belong to the same kind either. And that allows us to have an entity, such as a category, and then have our images under that. We have some different options for connecting to Datastore. For Java, we can use the DatastoreService API, which is the best performing method for Java. Or we can use JDO and JPA which will allow us to have a bit more portable code. And for Python we can use the NDB module; which is an ORM like library. And we can also use the G Cloud library. NDB is great because it allows us to interact with Datastore with very minimal amounts of code.

Let's check out the NDB models that we're using in our code. In the models .py file, we'll start by importing the NDB library. and then we create a class based on the NDB model. This is a Datastore entity that represents some metadata for our app. We're using it to determine if the application was initialized. The initialization process seeds the app with the initial images and data. And if we jump down to the category model, we can see that it has a name property, and a few class methods that we used to fetch some Datastore records. And then we have this image model, which has a URL, details, and tags properties. We'll be looking at these models further, later on. But for now, let's just check out how they're used by some of the code. Let's go back to our form submission and see how we create an image entity.

Okay, we can see here, we're passing in some key word arguments. The second one is the URL, the third one is the details, and those are the properties that we set on our model. And then we have this one here, it's the parent argument. This allows us to attach an image to a category. This creates our entity group. Now, we've created our entity and attached to the category and then we save it here with the put method.

This will return a key object that we could use to do future look ups. And if we need to display the key to our end users, maybe as some part of the front end code to look up a record, we can use this URL safe method on the key. That's what this is here. In the URL, after the slash images path, that's the images URL safe key. It allows us to look up a specific image object from Datastore. Now, there are a lot of different property types. So, we're not limited to what we can store.

Alright, let's look back at the image model, and check out the tags property. Because it's a bit different than the others. This string property ends up being a list because of this repeated equals true param. So this allows us to store a list of strings. We looked at the code to populate tags briefly, when we talked about the service account keys with the vision API. Tags get populated by the task that kicks off after the image is uploaded, and it uses the vision API to get back some labels, and then saves them as our tags.

Let's look back at the code in the pars image .py file. This is a modified version of the sample code that Google provides. The changes I made were to make it use the file from Cloud Storage and return a list of the labels that we'll use for our image tags. Let's see what this looks like when we store it in a list in Datastore. Here we have a tags column and it just looks like a basic array.

Now if we drill into this object, we can see here that we're using JSON to store the values. And we can filter on these as well. If we filter on this, lets say we try and query, where the tag field is equal to puggle, and we get one record. So if we match any value in that list we get a result back. Let's try querying something else.

If we query for food, and there we go. And if we check this entity out further, we can see it also has some additional tags. So we're not limited to single value properties; which is great for being able to model our entities without being limited.

Okay. Let's summarize what we've covered in this lesson. App Engine has access to different storage options, depending on what we want to do. Cloud Storage will allow us a nearly infinitely scalable blob storage option. Cloud SQL is a Google managed version of MySQL, that will allow us to use relational databases that we don't need to worry about managing. And Cloud Datastore is a native cloud NoSQL database. And we'll be diving into it a bit more in the next lesson.

So, if you're ready to learn more about the inner workings of Cloud Datastore then let's get started.

About the Author

Students38174
Courses17
Learning paths15

Ben Lambert is a software engineer and was previously the lead author for DevOps and Microsoft Azure training content at Cloud Academy. His courses and learning paths covered Cloud Ecosystem technologies such as DC/OS, configuration management tools, and containers. As a software engineer, Ben’s experience includes building highly available web and mobile apps. When he’s not building software, he’s hiking, camping, or creating video games.