After you’ve deployed Azure resources, such as virtual machines, you’ll want to monitor them to make sure they’re performing properly. You’ll want to be notified if any of them show signs of failure, such as taking a long time to respond or not responding at all. You’ll also want to discover if you should make any changes to them so they’ll operate more efficiently.

With so many potential issues to watch out for, it’s best to automate as much as possible. That’s why you need to use Azure Monitor. You can configure it to automatically collect data about your resources, analyze it, and notify the appropriate people when there are issues.

Azure Monitor automatically collects some information without requiring any configuration. Let’s have a look at a storage account to see what data it makes available. If you go down to the Monitoring section of the menu, the first option is called “Insights”. It shows you some graphs about things like transactions, availability, and used capacity. By default, it shows you data for the last four hours, but you can change that to another time period if you’d like.

By the way, some of Microsoft’s tutorials say that you should register these two resource providers before you start configuring Azure Monitor: Microsoft.insights and Microsoft.AlertsManagement. Although these resource providers do need to be enabled, in most cases, you won’t need to register them manually because Azure registers them automatically when you start using them.

All of the details on the Insights page are called metrics. They’re always numeric. For example, the used capacity might be 30 megabytes. Azure Monitor stores this data in a metrics database.

Azure Monitor also stores another kind of data: logs. They’re different from metrics because they typically contain text messages. They tell you about events that occurred, such as errors. 

If you click on the Failures tab, you’ll see error metrics for this storage account. If you want to understand why it’s getting errors, you can look at the logs. There’s a “Show Logs” button down here. But if we click the button, it says that this storage account isn’t storing log data because you need to configure logging first.

Before we do that, I’ll show you a log that is created by default. It’s called the activity log. If we click on it in the menu, it brings up the log entries for this storage account from the last 6 hours.

The activity log contains all of the subscription-level events about your resource rather than events that happened within the resource. So, it has an entry showing when the storage account was created, but it doesn’t show any errors related to data inside the storage account.

Thankfully, the activity log is free, so it doesn’t incur any charges. The downside is that it only keeps log entries for 90 days. If you want to keep them for longer than that, then you’ll need to route the log entries to go to another location, such as Azure Storage, and you’ll have to pay for the storage costs if you do that.

Another option is to send them to a Log Analytics workspace, which is a component of Azure Monitor Logs. They can be stored there for up to seven years. Although it’s significantly more expensive than Azure Storage, the advantage is that you can run sophisticated queries on your logs. And it’s not limited to activity logs. You can send all kinds of different logs to a workspace.

In fact, if we go back to the Failures tab in Insights and click “Show Logs”, it says that we need to create a Log Analytics workspace. Then it says that we need to enable diagnostic settings for our storage account and send the logs to that Log Analytics workspace. So let’s do that.

If you click on this link, it’ll open a new browser tab and take you to the right place. Create a new resource group and call it “DemoRG”. Call the workspace “LogWorkspace”. It doesn’t really matter which region you put it in, so we’ll leave that. Then click “Review + Create” and “Create”.

Now that we’ve created the Log Analytics workspace, we need to do the second step it mentioned, which is to configure the diagnostic settings to use this workspace. So we’ll go back to the storage account and click on “Diagnostic settings”.

We can configure separate diagnostic settings for each of the storage types in the account, that is blob, queue, table, and file, or the whole storage account. However, even though it looks like configuring the whole account would configure all of the storage types as well, that’s not how it works. If you want your logs to include entries for read, write, and delete operations, you’ll need to configure the individual storage types.

I’m only using blobs in this storage account, so I’ll click “blob”. Then click “Add diagnostic setting”. You have to name the setting, so just call it “blob”. Then choose which operations you want to track. Let’s include all read, write, and delete operations.

You can also choose to store transaction metrics if you want. Each blob storage request, such as a read or write request, is considered one transaction. Remember that metrics are numeric, so this wouldn’t store the actual transactions but would just keep a count of them. Let’s check it too.

Now we need to tell it where to send the data. We want to send it to the Log Analytics workspace we just created, so we’ll check that box, but it’s worth noting that you can also send the data to another storage account, an event hub, or a partner solution (meaning a third-party monitoring solution, such as Datadog or Dynatrace).

Okay, check the Log Analytics box. This is the workspace we just created, so we’ll select it. Then click “Save”. Now the logs for blobs in this storage account should start being sent to our Log Analytics workspace. However, it can take up to 90 minutes before that happens, so I’m going to switch to another storage account that already has logging set up.

All right, now when I click on “Show Logs”, it lists log entries that show errors. You can see that it gives quite a few details about each error. For lines that have a number bigger than 1 in the Error column, that means there are multiple entries that are very similar. So this line is summarizing 7 entries that are all basically the same.

If you want to do more log analysis, you can click on Logs. This is where we can write queries to search for specific details in the logs. To make it easier, Microsoft includes some example queries you can use. For example, this one lists the most common errors over the last 3 days. Click the “Run” button to see the query results. It shows two kinds of errors: ContainerNotFound and BlobNotFound.

We can also see the query it used to get these results. Log Analytics uses the same query language as Azure Data Explorer. It’s pretty easy to read. It starts off with the database table we want to query. In this case, it’s called “StorageBlobLogs”. Then it has a pipe symbol because we’re piping the data from the table into code that will perform an operation on it.

First, it says to only select data where the TimeGenerated if greater than 3 days ago. That might sound like it means data that’s older than 3 days ago, but it actually means data that’s newer than 3 days ago. And it also says to only include entries that do not contain “Success” in their StatusText. The exclamation point means “not”. We want to see operations that were not a success because we’re looking for errors.

The next line pipes the data to a “summarize” operator, which counts the number of instances of each unique status text. In this case, it found two unique statuses: ContainerNotFound and BlobNotFound.

The final line selects the top 10 most common statuses, and sorts them in descending order by the count for each. Since it only found 2 statuses, it didn’t need to select the top 10, but it did sort them in descending order because ContainerNotFound is the most common one.

If you’d like to write your own queries, you can edit one of the examples like this one, or you can read more about the query language at this URL, which you can find in the transcript below.

If you come up with a query you’d like to run on a regular basis, then you can save it. But you might also want to add it to a workbook, which is where you can display multiple charts showing metrics and log queries.

First, copy the query text, then click on “Workbooks”. Here’s an example of a workbook. Does this look familiar? It’s the same as the Overview tab in Insights. In fact, all of the Insights tabs are available here as workbooks. 

To create your own workbook, click “New”. Then click Add. You can add a few different items, but the main ones are log queries and metrics. Click “Add query”. Then paste the log query you copied. Then click “Run Query” to make sure it works. Good, it shows the same results as before. Now click “Done Editing” down here. It put the results at the top of the workbook. So, if we save this workbook, it’ll update this table every time we bring up the workbook.

We can also add a metric. You have to click “Add metric” again here. Then pick one, like “Used capacity”, and click “Save”. Let’s change the Time Range to “Last 24 hours”. That’s better. Then click “Done Editing”. Now we have both log query results and a metric graph in this workbook. We could add borders and headers and that sort of thing to make it look nicer, but you should have a good idea of how to create a workbook now. And you should also have a good understanding of how to configure metrics and logs in Azure Monitor, so that’s it for this overview.