Welcome back!

In this lesson, we’ll talk about device management, which will cover how to connect devices to IoT Hub by giving them an identity and credentials.

A device has a "functional" lifetime, which means that a device performs tasks, such as measuring temperature; and the tasks are specific to the devices purpose. They’re the devices reason for being.

Because multiple devices may perform the same task, you might them of them as group, rather than individually.

However, a device also has a "non-functional" lifetime, which means that a device may need to run some tasks to manage itself. These are task that aren’t a part of its core objectives. For example, maybe a specific device needs to undergo some sort of maintenance. Or maybe some devices need to receive updates.

Because of this non-functional lifetime, each device needs to have its own identity that allows it to be distinguished from other devices. And along with that identity it receives some credentials that allow it to communicate with IoT Hub.

IoT Hub contains an identity registry to store all the information about devices.

The registry allows you to create, update, get, delete and list identities. It also allows you to import and export identities using Azure Blob Storage.

All operations are available from a REST API so you can build your device workflow however works best for you. You can also use one of the SDKs, which I’ll show in a moment.

Each device identity contains some default information.

The DeviceId is the name you assign to a device.

GenerationId is the property used to distinguish devices with the same DeviceId, which were deleted and recreated.

So, the real key should be the DeviceId AND the GenerationId.

"auth" contain authentication information such as the symmetric keys, which contain a primary and secondary key shared with IoT Hub used to secure each message.
Those keys are encoded and stored in Base64 format.

Status, statusReason, and statusUpdateTime are used to enable or disable the device and trace why the device was disabled and when.

If disabled, the device cannot use its identity to access to IoT Hub.

ConnectionState and connectionStateUpdatedTime shows if a device is connected or not.
This property is available only if you use the AMQP or MQTT protocol and it is updated only every five minutes.

So, it could contain false positives and should be used only for debugging and testing purposes.
You need to implement some heartbeat functionality on your device to have a real feedback on a connection from your device in a production environment.

LastActivityTime parameters tracks the last operation from that device.

Device identity is secured by a couple of values.

First value is the device ID.

The second value is a shared access policy used to authorize the device to communicate with IoT Hub.

It’s based on a shared secret, a symmetric key, that is used to create an authorization token.

Every request comes with a token, so IoT Hub can recognize which device is sending that.

Only the device and IoT Hub know that policy. Every device has a different policy and key. And if you think that a secret is compromised you can regenerate the keys.
Let’s try working with the IoT Hub device registry through a basic command line application.

We’re here on the command line, and I have a basic application I’ve created that allows me to interact with the device registry, called IoT Hub Device Manager.

This app is a thin wrapper around the CRUD functions that the device registry provides. For example if I pass in the list parameter…you can see that we have some active devices at the moment.

If you want to see more information about a particular device you can use the "get" command, and you need to specify with the "deviceId" parameter. Again, this is just my own app based on the device registry’s existing functionality.

You can see that it shows the different status’ and shared-access-keys for the devices.

If you want to disable the device you can use the "disable" command, and even add in a reason with the "reason" parameter, which is a text string.

And if check the device again...notice it’s now it's disabled.

If you want to add a new device, use the "add" command, specifying the "deviceId" parameter, and you have your new device. If we look at the list of devices, you can see that there’s a new device.
Now let's see how all these commands are implemented, so you can see the underlying SDK.

We are here in Visual Studio 2015, in the solution, and the project is called IoTHubDeviceManager. It is written in C#, and it’s just a command line application.

This is using the helper library that I showed your earlier, so if you’re not sure where some of this code is coming from, download the code and your can check it out for yourself.

The code starts out by setting up the connection information, it needs the host name, which is the name of the IoT Hub instance we want to connect to.

It needs the shared access key name, which is the name of the policy used to connect, and then the key value

Being a thin wrapper around the device registry, I’m just using the first parameter as the command name, and you see that I have add, get, list, remove, enable and disable.

Every command implementation receives an instance on the registry manager, which is the object that you can see up here. You create an instance with the static method "CreateFromConnectionString" on the RegistryManager class.

The RegistryManager comes from the Microsoft.Azure.Devices library and contains all the functionality for interacting with the device registry.

We need to build the connection string with the parameters we supplied.

Then all commands receive a configuration object, and a list of arguments from the command line, so each command can retrieve its specific parameters.

Let’s check out the methods used by the Registry Manager that are used to handle the CRUD functions.

First up is the Add Device Async, which allows you to add a new device to the Device Registry.

Next up, is the Get Device Async which expects a device ID, and then will return an individual device from the registry.

You can also disable a device, and you can do that by getting the device, and then changing the status to “Disabled” and then using the Update Device Async method to send the updated status back to the registry.

Enabling a device is just like disabling, only you’ll set the status to disabled.

To list the devices from the registry you can use the Get Devices Async.
In this app, for each of a listed device, I output a bit of info, just to see some basic details. For example, the GenerationId, the DeviceId and the primary key of a secure token.

We can also remove a device by retrieving the device and then calling RemoveDeviceAsync passing the device object. Or you can pass in device ID.

So, there’s more code here for you to check out, however that covers the Registry Manager pretty well.

Okay, let’s wrap things up here, and summarize what we’ve covered so far.

Every device has an identity. You needed to generate an identity for each device you want to connect to an IoT Hub. Along with the identity, you give a device a symmetric key to securely send and receive messages.

All the identity information for all devices are stored in an identity registry inside IoT Hub.
In the next lesson, we will talk about communication between the device and the IoT Hub Service.

Alright, I’ll see you in the next lesson.

0040 - Device To Cloud
Welcome back!

In this lesson, I'll talk about device-to-cloud communication, which means how devices send data to IoT Hub.

Device-to-cloud sounds pretty self explanatory, however there are different ways to send data to IoT Hub. In this lesson, we’ll cover one way, which is device-to-cloud messages.

So let’s start by explaining some of the concepts of device-to-cloud messages.

A device-to-cloud message is a sort of envelope. It takes your telemetry and or alert data, and sends it as a blob it IoT Hub.

If you’re new to IoT, telemetry is the process of acquiring, measuring, and sending data from devices. Data can be an event, meaning that it’s a snapshot of some thing that happened at specific point in time. It could also be the device’s state at a point in time. Or it could be an alert, to indicate an action needs to be taken.
IoT Hub doesn’t care about the body of your message. Consider a package delivery service, such as FedEx. Their primary focus is getting your package to its destination. IoT Hub is FedEx in this example. It doesn’t care about what data you’re sending, it just needs to get it from the device to IoT Hub.

A message requires you to serialize your message body and then deserialize it at the destination.

Since your message body is serialized you won’t be able to access the data unless it’s deserialized. However you may want to access some properties without having to deserialize, and for that the message also allows you to set properties that are outside of the message body.

Besides having properties you can set, there are also system properties used to interact with IoT Hub. For example you can use the Ack property to request an acknowledgement message.

I’m not going to go through all of these, however it’s worth checking out the link on the screen to see the descriptions of all the properties.

When thinking about messages it’s important to consider the message size, which can’t exceed 256 kilobytes, and is more than the size of your message body.

To calculate the message size you need to consider the message body, plus the size of the dictionary of your custom properties, which includes the keys and values; you also need to include the set of system properties, just the values though, not the keys.

After a device has sent a message to IoT Hub, the message is received by IoT Hub from a device endpoint and directed to an EventHub-compatible service endpoint.

Each device has an endpoint where it can send its message using either HTTP, AMQP, or MQTT.

IoT Hub guarantees readability and durability when handling the messages, to help in situations where there can be intermittent connectivity on the device side.
It implements "at least once delivery" for messaging from the device.

When the message arrives, it is managed in IoT Hub with a transient approach. You can configure how long IoT Hub should keep the message, from the default of one day, up to seven days. After the timeout window that message is treated as a dead message.

So, you'll need to decide how much time you need to handle all the messages from a failure point of view.

Once a message is stored in a message queue, they’re then available through an Event Hub compatible interface. Because of that, you have a lot of options for reading the messages.

You could create an Event Hub event processor, with one of the EventHub SDKs. Which is probably the most flexible approach, but also the most complicated.

You can also use Azure Stream Analytics, which gives you the expressive power, and familiarity of a SQL-like language to manage events with a, time window based approach.

You can use Azure WebJobs, or you could use Azure Functions, which is Azure’s serverless approach.approach.

Let’s check a demo showing data flowing from simulated devices to IoT Hub, and then the messages will be picked up by an Azure function, and saved to DocumentDB without any processing.

For this demo, I’ve prepared a PowerShell script which has some commands ready to launch.

I mentioned previously that in this course we’ll be using a device simulator rather than a device, and that’s because it’s much easier to work with in development, as well as in demos.

So, I have a command line application here written in C#, that uses the same SDK that devices use.

Since we have no real sensors generating data, we’ll simulate the data by generating random values as a fake sensor, and then we’ll build a fake event message based on the values.

This simulator that I’ve named IoT Hub Device Simulator, has an option for "fake telemetry" that runs an endless loop, sending continuously fake telemetry data.

In this case, you can see temperature data that starts from an initial value of -20°C, and then sends out randomized changes.

On line 18 I’m doing the same thing, except with humidity, instead of temperature.

On line 15, I’m passing in the event type of "telemetry", which allows for a single event to be sent, rather than an endless loop of the fake telemetry data.

And finally I have a more generic command named "send event," which isn’t a data generator, rather it allows you to create the event yourself, by specify the JSON object.

Let’s run the first command here, and see the simulator generate some fake temperature data.

Notice at the top of the console the device ID is North3, and it also lists off its shared key. For this I’m using MQTT.

It’s designed to write the JSON to the console whenever it randomly generates an event. This data is also being sent to IoT Hub, which we’ll check out shortly.

Let’s also fire up another processes, using some different parameters.

This will simulate another device, called South2 that generates fake humidity telemetry events with a different frequency.

Okay, let’s see how this is implemented in code.

If we go to Visual Studio 2015, there is the IoT Hub Device Simulator project.

There is a Device Simulator.cs file with a Main function.
The structure is always the same, it uses the command line to retrieve the hostname, the deviceId, the Shared Access Key, and the transport type.

The interesting part here is the selection of the authentication method.

You start from a deviceID and the shared access key and using the static method on the authentication method factory, you can build an authentication object.

This is important because when you create a Device Client object by invoking the static method deviceclient.create it requires arguments for, the host name, authentication method, and transport type.

Then we have the common parameter that is the first parameter we set on the command line.

Here is where the different commands are implemented.

For example, we ran "fake telemetry" and if we follow the definition, you can see that we parse the args array to obtain the specific parameters. For example, to have the value of a delay for the sample generation.

The values are generated in an infinite loop here, with a sleep function to delay the generation. And then you have a method "Telemetry Async".
Following the definition, the methods starts by setting the telemetry type to “temperature.”

Then we build out the telemetry event, serialize it to JSON, and then send it to IoT Hub with the Send Event Async method.

The SendEventAsync is just another helper method. I’m using this method to build the real Message for IoT Hub using the specified event body and converting it to a Byte array.

I’m also adding a message property with the key of "type" which will store the type of event this is, and we’ll use this in another lesson.

Then to actually send this to IoT Hub I invoke deviceclient.SendEventAsync and pass in the message.

The other methods work similarly to the Telemetry Async method.

There’s a lot of boilerplate code here, but it’s the idea behind it is that this would have some reuse, rather than creating a simulator for each lesson and purpose.

Well, the simulators are still sending events to IoT Hub service, however, where’s all this data going after it’s queued up with IoT Hub? What are we actually doing with the messages?

I’ve implemented a function in a Functions App, that I call "Handle Telemetry", and it’s triggered by the message delivered to an EventHub, since IoT Hub implements an Event Hub compatible endpoint.

The function is very basic! All it does is store an event in a document inside of DocumentDB.

Using a DocumentDB database is among the simplest ways we can store the data, because we can store the data directly without any change, since it’s already JSON data, and then we can query as needed.

Since processing the data isn’t the focus of the course, using Azure Functions and DocumentDB gives you just a glimpse into the data being sent from the simulated devices. If you’re new to Functions or DocDB, I recommend checking out our courses for each of those services.

This code is quite simple because we take the event from parameter and we suppose it is already in JSON format, and then send it directly to an output Document parameter to store the document on DocumentDB.

If we look at the "logs" window in the bottom of the Functions page and I clear it, you can see that the log is still growing, so the function is still running. It’s being invoked because the simulators are running.

So, how do we see all the events that are stored?

For that, we can check out the DocumentDB collection. Going through the document explorer, you can see that it’s full of documents. And if you want to query them, you can use the query explorer. Notice here that you have a query, "Select * From c," and if we run it, we see the events. For example ...the temperature data is here, and if we scroll down, you can see that the humidity data.

If I add a where condition to the query, “c dot telemetry type equals humidity”, and I rerun the query, you have only the humidity values.

And if we add another clause to the filter, let’s say, and C dot current greater than 21.
21 is just an arbitrary value, but is shows how powerful DocumentDB can be for storing this sort of data.

Let’s check IoT Hub from the portal next, notice we have a dashboard with some messages, and the message counter keeps increasing, it’s not real time, however, if I refresh the page, notice it’s changing, because the simulators are still running.

If we go into "Shared Access Policies" you’ll see the Handle Telemetry policy that I have created for the Function to allow it access to the events, and you can see the connection string with a primary key that you use to connect to Azure Function.

Okay, we covered a lot in this lesson so let's do a quick recap.

In this lesson we talked about how IoT Hub doesn’t care about the contents of your messages. Rather it cares about the system properties, and getting your message into an Event Hub compatible message queue.

We also talked about how IoT Hub doesn’t do anything with your messages except hold them in aqueue. Which means it’s up to you to process the messages.

In the next lesson, we will continue to talk about device-to-cloud communication, and in particular endpoints and file uploads.

Alright, I’ll see you in the next lesson!