Introduction to Cosmos DB
Introduction to Using Cosmos DB
Introduction to Creating an App with Cosmos DB
The course is part of these learning pathsSee 4 more
Cosmos DB is one of many database solutions in a crowded market. From DynamoDB to Cassandra to CockroachDB, the questions one would naturally ask when examining Cosmos DB are, “what makes it special and how can I get started with it?”
This course answers both of those questions as thoroughly and concisely as possible. This course is for anyone with an interest in database technologies or creating software using Microsoft Azure. Whether you are a DevOps engineer, a database admin, a product manager, or a sales expert, this course will help you learn how to power your technology with one of Azure's most innovative data solutions.
From this course, you will learn how to make use of Cosmos DB's incredible flexibility and performance. This course is made up of 9 comprehensive lectures to guide you from the basics to how to create your own app using Cosmos DB.
- Learn the basic components of Cosmos DB
- Learn how to use Cosmos DB via the web portal, libraries, and CLI tools
- Learn how to create an application with Cosmos DB as the data backend
- People looking to build applications using Microsoft Azure
- People interested in database technologies
- General knowledge of IT architecture
- General knowledge of databases
Okay, so now that we have a working Cosmos DB service and backend running and we have an Event Hub for receiving events and messages, the next thing we need is a Azure Function. That's gonna be our actual service for processing the data we receive and we have an example here called bethune, and we'll dig into the code and a little bit but before we do that, I wanna show how it is we set up an Azure Function app.
As with a lot of Azure services, there's two parts essentially. There's the shell, the Function app. And then there's the code itself. The function that's actually running. So as before we can just go through the create a resource wizard or Function app. We click Create. Pay as you go is fine. We want an existing resource group. We'll put it in bethune2. We'll name it bethune2 as well.
There's some additional configuration here. We go through our hosting. It's gonna need a storage account so we'll just, there isn't an existing one in this namespace, in this account so we'll create a new one by default. We'll just call it new storage account, bethune, whatever, that's fine. Operating system, Windows is okay. We'll stick with the Windows default. And our plan type is serverless. We don't need to pick one of the options, other options here.
There's an app service plan and premium plan. If you know you need, you know, more storage and more resources. And then monitoring we can accept, well actually we'll go with no. We don't need to enable application insights. That's more if we need more deep level granular monitoring. You can add that if you'd like, but for this tutorial we won't need it. And for tags, we're not gonna add any tags. That's fine.
So we'll go ahead and review and create. And as always, it'll do a validation first. Config seems to be okay so we click Create and we'll just give this a few moments and we will have our bethune2 Function app. Right? There we go. Deployment in progress, let's give that a minute. Okay, all right.
So eventually the deployment will create, will complete and it shouldn't take more than a minute or so. So once it's finished, we can just click Go to resource, or we can find it from the dashboard. And you'll see we now have a bethune2 Function app, and this is our sort of our container for our serverless Azure Function, our actual code.
So right now there's not really any code executing, there's just this account. So what we have to do is click on Functions and here is where we can go ahead and create our event processor. So we'll click on Add, and this will take us through a pretty simple wizard to set something up.
Now, when you click New Function here, it's gonna ask for the type of function, there's a number of templates here. HTTP trigger, Timer trigger. These are the conditions that will determine when the code executes. So there's many types of triggers here. For our purposes, we're interested in this one, the Azure Event Hub trigger.
So with an Azure Event Hub trigger, what'll happen is the code for the Azure Function will execute whenever a message or an event is received by that event hub. So we'll go ahead and we'll call this EventHubTrigger2, and then we will come up with some connection. We'll have to create a new connection here.
So in order to create a connection, you'd need an existing event hub. Now fortunately we have an existing one. We just created the bethune2 event hub a little while ago. So we'll use that. It'll put in the rest of the config automatically. We'll call it bethune2 connection. We'll use the route manage access the route manage shared access key policy for that. Click OK. And now we have an event hub trigger for the event hub we created.
Event hub name, we'll go ahead and call this bethune2 and we'll give it the default consumer group. As you know, event hubs can have multiple consumer groups. So we click Create Function and this will, again, it's not gonna give us the code for the function itself, but it will give us the configuration for the trigger. And that's key for getting everything to be wired together properly. So just give this a second.
We can refresh it after a while and we should see, boom, there it is. There is our function EventHubTrigger2. And if we click on that, we can see here that the basic configuration, there's a code here and there's the integration.
Now the integration, we'll start with that. This is defined by our trigger, by the type of function. Then we selected Event Hub. So we can see here eventHubsMessages is our trigger. And then this is where the actual function code is. I'll come to that in a second here. But if we look at the trigger here, this is the configuration that we just set. And we can actually look at this as code in a little bit.
So we see here it's a bindings config type eventHubTrigger. name, eventHubMessages, direction in. So this is, you know, if a message comes into the event hub, it'll trigger this. Event Hub name, bethune2. The one we created and here's our connection. Cardinality, many. I'll come back to that in a bit, it deals with whether we're dealing with individual messages or arrays of objects. And we care about the default consumer groups. So this function.json file is very important. This is our configuration for integrating with Event Hub.
There's a little for-loop here. It'll go through every message in this object. It's assuming that it's receiving an array of messages. See, there's that cardinality thing. It's assuming that there's multiple messages here. So we can do a test run of this if we'd like. We can just do Test and Run and click Run, and we can see it'll connect to our function. And if we click Run we'll see, it should just output some basic message text.
So what we're gonna do is we're gonna go back home and we're gonna go back to our Cosmos DB account, bethune2, and we're gonna go into the Data Explorer, and we're gonna set up a container that will give us a collection within the database that we can use.
Now here in the Azure Function configuration, the main thing we need to do, if we go here we can say that, that was not set. We'll go back to the integration and we can see here there's no outputs defined. This is the main goal. We have to define an output so that the function knows to save its results or its output somewhere, and that's going to be Cosmos DB.
Now what we need to do is we need to create a container within our bethune test DB. So we'll click New Container. We're gonna use this existing database and the container ID will be bethunecontainer. We'll just call it bethune, we'll call it bethune2container, why not. And then we will also give this a partition key. Let's just call this testkey1, simple enough. So we have a partition key. This will help us ensure things work the right way. So click OK. And very quickly we will have a collection that we can use for storing the output of our Azure Function. All right, perfect.
So that's already set and good to go. And if we click on the collection we'll see there's no data in it. There's no items there. And hopefully if we do a test, we'll see items show up. So let's go back to our event hub and our Azure Function here, and we will add an output for this new collection.
Now the output, we have to pick Cosmos DB, that's the output type we want. And then we get a few interesting bits of configuration here. This output document, that's the parameter that we're gonna care about in the actual code. And then the database name. We wanna use this bethune test database so we'll put that in. And the collection we want to use would be the same as this container name, bethune2container.
One of the nice things is there's this feature here called if true, create if doesn't exist, basically. It means that if you don't have this container or if you don't have this database or collection existing, then the function will create it for you. So we can leave that as yes, you can leave it as no if you don't want it automatically doing that. But what we have to also do is create a connection to the Cosmos DB account.
What we'll do is we'll click New and want to connect to the bethune2 database. We click OK. And we'll have this connection here. Actually they're already is an existing connection we can use, but in any event, in this part here, as long as you already have a Cosmos DB account, you should be able to just find, fill it in here and put in your connection. And then for the partition key, the partition key we had before here in this container we saw was testkey1. So we can literally just copy that and paste it and click OK.
So now we have an output. Now we actually have an output for our function. So how can we test this? How can we make sure that it works appropriately? So we're not gonna test the event hub piece. We just want to test that the function can output into Cosmos DB. Well, there's a very simple way to test that.
We've added this output section for the Cosmos DB integration. We have the parameter name for the output document here for what will actually go into Cosmos DB. We have the config for the database name, the collection, create if not exist. The string for the connection and then the partition key and the type direction. So this has all changed. And of course here, the cardinality has changed.
So we're taking in a single object. We have this processed output. And then this line is important. We have here the output document parameter. This is what's gonna go into Cosmos DB. So we call this output document and we call a JSON.stringify to make sure we get a string. And this is the thing that will go into Cosmos DB. So what we'll do is we will save this and we will do a quick test run to see if it works.
So we'll click Test. Now if we go into Cosmos DB now we can see in the bethune2container collection. If we click on items there's nothing there, right? There's no items for us now. So if our code works, we should see something show up there. So we have a little silly message there, a test message banana face. We know for sure that that's us. And so we just run, click Run here and cross fingers. Hopefully we should see this execute without any failures. And there we go.
Okay it executed. And now let's see if this actually went into Cosmos DB, we'll click on Items. And there it is, there's our message, Test Message banana face. So we know that our function works in terms of the output piece. We know that we can have the Azure function connect to Cosmos DB and save messages in a collection of our choice in the database of our choice.
Now the last thing we need to do is test the event hub piece. We wanna see the whole thing work end to end. We want to see a message go into event hub, and then that trigger the Azure function. And then after that function is triggered, it should then save it into Cosmos DB. We know this Cosmos DB part works, but we wanna see the event hub part as well.
So that's what we'll do in the last section of this demo. It should be fun.
Jonathan Bethune is a senior technical consultant working with several companies including TopTal, BCG, and Instaclustr. He is an experienced devops specialist, data engineer, and software developer. Jonathan has spent years mastering the art of system automation with a variety of different cloud providers and tools. Before he became an engineer, Jonathan was a musician and teacher in New York City. Jonathan is based in Tokyo where he continues to work in technology and write for various publications in his free time.