Welcome back!

In this lesson, I'll talk about device-to-cloud communication, which means how devices send data to IoT Hub.

Device-to-cloud sounds pretty self explanatory, however there are different ways to send data to IoT Hub. In this lesson, we’ll cover one way, which is device-to-cloud messages.

So let’s start by explaining some of the concepts of device-to-cloud messages.

A device-to-cloud message is a sort of envelope. It takes your telemetry and or alert data, and sends it as a blob it IoT Hub.

If you’re new to IoT, telemetry is the process of acquiring, measuring, and sending data from devices. Data can be an event, meaning that it’s a snapshot of some thing that happened at specific point in time. It could also be the device’s state at a point in time. Or it could be an alert, to indicate an action needs to be taken.
IoT Hub doesn’t care about the body of your message. Consider a package delivery service, such as FedEx. Their primary focus is getting your package to its destination. IoT Hub is FedEx in this example. It doesn’t care about what data you’re sending, it just needs to get it from the device to IoT Hub.

A message requires you to serialize your message body and then deserialize it at the destination.

Since your message body is serialized you won’t be able to access the data unless it’s deserialized. However you may want to access some properties without having to deserialize, and for that the message also allows you to set properties that are outside of the message body.

Besides having properties you can set, there are also system properties used to interact with IoT Hub. For example you can use the Ack property to request an acknowledgement message.

I’m not going to go through all of these, however it’s worth checking out the link on the screen to see the descriptions of all the properties.

When thinking about messages it’s important to consider the message size, which can’t exceed 256 kilobytes, and is more than the size of your message body.

To calculate the message size you need to consider the message body, plus the size of the dictionary of your custom properties, which includes the keys and values; you also need to include the set of system properties, just the values though, not the keys.

After a device has sent a message to IoT Hub, the message is received by IoT Hub from a device endpoint and directed to an EventHub-compatible service endpoint.

Each device has an endpoint where it can send its message using either HTTP, AMQP, or MQTT.

IoT Hub guarantees readability and durability when handling the messages, to help in situations where there can be intermittent connectivity on the device side.
It implements "at least once delivery" for messaging from the device.

When the message arrives, it is managed in IoT Hub with a transient approach. You can configure how long IoT Hub should keep the message, from the default of one day, up to seven days. After the timeout window that message is treated as a dead message.

So, you'll need to decide how much time you need to handle all the messages from a failure point of view.

Once a message is stored in a message queue, they’re then available through an Event Hub compatible interface. Because of that, you have a lot of options for reading the messages.

You could create an Event Hub event processor, with one of the EventHub SDKs. Which is probably the most flexible approach, but also the most complicated.

You can also use Azure Stream Analytics, which gives you the expressive power, and familiarity of a SQL-like language to manage events with a, time window based approach.

You can use Azure WebJobs, or you could use Azure Functions, which is Azure’s serverless approach.approach.

Let’s check a demo showing data flowing from simulated devices to IoT Hub, and then the messages will be picked up by an Azure function, and saved to DocumentDB without any processing.

For this demo, I’ve prepared a PowerShell script which has some commands ready to launch.

I mentioned previously that in this course we’ll be using a device simulator rather than a device, and that’s because it’s much easier to work with in development, as well as in demos.

So, I have a command line application here written in C#, that uses the same SDK that devices use.

Since we have no real sensors generating data, we’ll simulate the data by generating random values as a fake sensor, and then we’ll build a fake event message based on the values.

This simulator that I’ve named IoT Hub Device Simulator, has an option for "fake telemetry" that runs an endless loop, sending continuously fake telemetry data.

In this case, you can see temperature data that starts from an initial value of -20°C, and then sends out randomized changes.

On line 18 I’m doing the same thing, except with humidity, instead of temperature.

On line 15, I’m passing in the event type of "telemetry", which allows for a single event to be sent, rather than an endless loop of the fake telemetry data.

And finally I have a more generic command named "send event," which isn’t a data generator, rather it allows you to create the event yourself, by specify the JSON object.

Let’s run the first command here, and see the simulator generate some fake temperature data.

Notice at the top of the console the device ID is North3, and it also lists off its shared key. For this I’m using MQTT.

It’s designed to write the JSON to the console whenever it randomly generates an event. This data is also being sent to IoT Hub, which we’ll check out shortly.

Let’s also fire up another processes, using some different parameters.

This will simulate another device, called South2 that generates fake humidity telemetry events with a different frequency.

Okay, let’s see how this is implemented in code.

If we go to Visual Studio 2015, there is the IoT Hub Device Simulator project.

There is a Device Simulator.cs file with a Main function.
The structure is always the same, it uses the command line to retrieve the hostname, the deviceId, the Shared Access Key, and the transport type.

The interesting part here is the selection of the authentication method.

You start from a deviceID and the shared access key and using the static method on the authentication method factory, you can build an authentication object.

This is important because when you create a Device Client object by invoking the static method deviceclient.create it requires arguments for, the host name, authentication method, and transport type.

Then we have the common parameter that is the first parameter we set on the command line.

Here is where the different commands are implemented.

For example, we ran "fake telemetry" and if we follow the definition, you can see that we parse the args array to obtain the specific parameters. For example, to have the value of a delay for the sample generation.

The values are generated in an infinite loop here, with a sleep function to delay the generation. And then you have a method "Telemetry Async".
Following the definition, the methods starts by setting the telemetry type to “temperature.”

Then we build out the telemetry event, serialize it to JSON, and then send it to IoT Hub with the Send Event Async method.

The SendEventAsync is just another helper method. I’m using this method to build the real Message for IoT Hub using the specified event body and converting it to a Byte array.

I’m also adding a message property with the key of "type" which will store the type of event this is, and we’ll use this in another lesson.

Then to actually send this to IoT Hub I invoke deviceclient.SendEventAsync and pass in the message.

The other methods work similarly to the Telemetry Async method.

There’s a lot of boilerplate code here, but it’s the idea behind it is that this would have some reuse, rather than creating a simulator for each lesson and purpose.

Well, the simulators are still sending events to IoT Hub service, however, where’s all this data going after it’s queued up with IoT Hub? What are we actually doing with the messages?

I’ve implemented a function in a Functions App, that I call "Handle Telemetry", and it’s triggered by the message delivered to an EventHub, since IoT Hub implements an Event Hub compatible endpoint.

The function is very basic! All it does is store an event in a document inside of DocumentDB.

Using a DocumentDB database is among the simplest ways we can store the data, because we can store the data directly without any change, since it’s already JSON data, and then we can query as needed.

Since processing the data isn’t the focus of the course, using Azure Functions and DocumentDB gives you just a glimpse into the data being sent from the simulated devices. If you’re new to Functions or DocDB, I recommend checking out our courses for each of those services.

This code is quite simple because we take the event from parameter and we suppose it is already in JSON format, and then send it directly to an output Document parameter to store the document on DocumentDB.

If we look at the "logs" window in the bottom of the Functions page and I clear it, you can see that the log is still growing, so the function is still running. It’s being invoked because the simulators are running.

So, how do we see all the events that are stored?

For that, we can check out the DocumentDB collection. Going through the document explorer, you can see that it’s full of documents. And if you want to query them, you can use the query explorer. Notice here that you have a query, "Select * From c," and if we run it, we see the events. For example ...the temperature data is here, and if we scroll down, you can see that the humidity data.

If I add a where condition to the query, “c dot telemetry type equals humidity”, and I rerun the query, you have only the humidity values.

And if we add another clause to the filter, let’s say, and C dot current greater than 21.
21 is just an arbitrary value, but is shows how powerful DocumentDB can be for storing this sort of data.

Let’s check IoT Hub from the portal next, notice we have a dashboard with some messages, and the message counter keeps increasing, it’s not real time, however, if I refresh the page, notice it’s changing, because the simulators are still running.

If we go into "Shared Access Policies" you’ll see the Handle Telemetry policy that I have created for the Function to allow it access to the events, and you can see the connection string with a primary key that you use to connect to Azure Function.

Okay, we covered a lot in this lesson so let's do a quick recap.

In this lesson we talked about how IoT Hub doesn’t care about the contents of your messages. Rather it cares about the system properties, and getting your message into an Event Hub compatible message queue.

We also talked about how IoT Hub doesn’t do anything with your messages except hold them in aqueue. Which means it’s up to you to process the messages.

In the next lesson, we will continue to talk about device-to-cloud communication, and in particular endpoints and file uploads.

Alright, I’ll see you in the next lesson!