Hello and welcome, this is Introduction to Cloud Firestore with App Engine, and in this lesson, we'll be exploring the basic functionality of Cloud Firestore and how it's used with App Engine. By the end of this lesson you should be able to: explain the purpose of Cloud Firestore, explain the relationship between Cloud Datastore and Cloud Firestore, explain when to choose Datastore Mode, and explain how to use Datastore with App Engine. 

Most applications need a place to store queryable data, and fortunately, there is no shortage of databases to help. However the choice of database is not always a simple one, it depends on multiple factors including but not limited to the type of data being stored, how the data will be consumed, et cetera. Technical limitations are another concern. For example, will the database be able to support the amount of traffic? 

No matter what kind of database you're using, if it can't support the workload that you intend to use it for, it's not all that useful. Remember, App Engine can keep scaling up to support the demand and that means the database needs to as well, and that's why Google created Cloud Datastore and paired it with App Engine. Now, that doesn't mean it's the only database you can use though it was designed to be a good likely pairing for many apps. So, if you're interested in learning more, then let's get started. 

Google released Datastore in 2013 to serve as App Engine's database, and for years that is how it has been. Datastore is a highly scalable, fully-managed, no-SQL document database. Supporting both eventual and strong consistency, it supports transactions and it offers a SQL-like query language called GQL. It stores data as properties of an entity with support for multiple data types, and it categorizes entities based on a developer supplied Kind. 

To make entity lookups perform quickly, entities include a Datastore property named a Key which is a unique ID, and to allow entities to be queried, Datastore allows developers to create indices based on the properties for which we want to filter. 

I mentioned queries being similar to SQL, they're similar but not exact, and there are some limitations for which I'll include a link App Engine is designed to run web apps and mobile backends which are broad categories with a wide range of storage requirements. There's really just no single option that's going to work for all workloads. If we were to compare a mobile chat application with a mobile news feed, both of them have very different storage needs. The same goes for web applications, let's say a brochureware application is going to have different storage needs than a site such as Twitter. 

In 2014, Google acquired a realtime database called Firebase. Realtime databases are used for data which is always changing and require processing to happen very quickly. Shortly after that acquisition, Google started building Cloud Firestore. It took the best parts of Firebase, it took the best parts of Datastore, and it smashed them together into a single service. 

Now, the purpose of Cloud Firestore is to serve as the next generation no-SQL database for Google Cloud. This is where we get a bit tangled. Cloud Firestore provides one of two operating modes that are called Native Mode and Datastore Mode. Using Cloud Firestore in Datastore Mode is the next generation of Cloud Datastore. It supports Datastore's API and it uses Firestore's data storage layer which removes some of the previous Datastore limitations. 

Now, currently the original Datastore and Firestore in Datastore Mode are two different products, however, Google will be moving users over to Firestore seamlessly over time. Using Firestore in Native Mode is not the next generation of Firebase, at least not publicly, at least not in this moment. Using Cloud Firestore in Native Mode is similar to Firebase, though there are some implementation changes. 

So with all of this, the logical question is, how do I know which mode to use? Here's some general advice, now, it's not universally applicable, though it is a good starting place. Datastore or Firestore in Datastore Mode are intended for server workloads, meaning a service-side application interacts with a database. Now, this is exactly what App Engine does, so if you're building with App Engine and you need a schema list database, then you'll probably wanna use Firestore in Datastore Mode. Firestore in Native Mode is similar to Firebase, which is basically an application platform and doesn't require developers to create service-side applications. 

Firestore Native and Firebase both provide STKs which allow read and writes to documents for which users have access and all without a server, and if you need some sort of backend functionality, it also supports running Cloud functions based on event triggers which include, when a document is created, deleted, updated, et cetera. So Firestore in Native Mode is very cool, but it's not the focus of this lesson, so we're not going to go into detail, this lesson is really focused on using Datastore Mode with App Engine, though if you wanna learn more, I'll include some links for further reading. 

Before moving on, there are some things to commit to memory, each project can use Firestore in one mode only. So if you select Datastore Mode, that's the mode for your project and vice versa with Native Mode. Also, and importantly, if you create an App Engine app inside a project, it's going to automatically choose Datastore Mode, so be careful with that, if you really don't wanna Datastore Mode, don't create an App Engine app in that project first. 

Alright, with all of this out of the way, how do you actually use Datastore with an App Engine application? Due to the ever-evolving nature of App Engine, there have been different methods for interacting with Datastore which varied by runtime. Currently, Google recommends the use of the Google Cloud Client Libraries. Google provides these libraries for different runtimes, and they allow engineers to interact with Datastore in the language that they're familiar with. If for some reason you're using a language that does not have a supported library, you can always fall back to the Rest API and use that directly, it just requires a bit more effort on your part. 

Before wrapping up, let's check out a demo of Cloud Datastore in the console. I'm here on the Datastore Dashboard, I've already enabled the Datastore API and I have some sample entities which store some dummy data. Notice here I have a Kind called EmailEvent and there are four entities. This Entities page is where we can see the different entities for a specific Kind which is specified in the dropdown. And if we click on Create it will open a form where we can add a new entity. 

The Namespace is used to partition data, an example might be to specify a company name which would allow for a multitenant app, and if you don't specify a namespace that's fine, the default is used, just know, they can't be changed once they're set. Datastore uses the concept of a Kind to categorize an entity and that makes it easier to query specific types of entities The Key is used to look up an entity quickly, notice these properties here, these are added by Datastore to make it easier for us to populate this, it knows that this is an EmailEvent Kind so it's given us the properties that exist on some of the other entities. 

This is just a nice feature from the user interface to make it easier to enter our data, remember, this is a schema list database, so we don't have to define specific properties for even the same Kind, I could just remove all of these and have a totally different property for one EmailEvent than I do for all the rest. I can't imagine a use case in which you'd want to do that, but it is possible. 

The query language that Datastore supports is called GQL and again, and it resembles SQL up to a certain point, so if we start by typing select star from followed by the name of our Kind, we can see this returns all EmailEvent Kinds with all the properties, though you could also specify properties to return and that way, you can get just the data you need. 

OK, let's stop here and see how we did. The purpose of Cloud Firestore is to serve as the next generation no-SQL database for Google Cloud. To get there, it took the lessons learned from Datastore and Firestore, probably other stuff internally as well, and put them all into a single product. Cloud Datastore is currently both its own service and a mode of Firestore, where Firestore in Datastore Mode is the latest generation of Datastore and will replace Datastore. Datastore Mode is intended for server workloads and if you're pairing with App Engine, then it's likely a good choice. To interact with Datastore or Datastore Mode in software, we would use the Google Cloud Client Libraries, though we could always use the Rest API directly if we needed to. 

Alright, that's going to do it for this lesson, I hope this has been helpful, I hope it's filled in a few of the gaps, and I will see you in the next lesson.