Scaling and Resilience
This course will show you how to design and create web applications using the Azure App Service.
By the end of this course, you'll have gained a firm understanding of the key components that comprise the Azure App Service. Ideally, you will achieve the following learning objectives:
- A solid understanding of the foundations of Azure web app development and deployment.
- The use cases of Azure Web Jobs and how to implement them.
- How to design and scale your applications and achieve consistent resiliency.
This course is intended for individuals who wish to pursue the Azure 70-532 certification.
You should have work experience with Azure and general cloud computing knowledge.
This Course Includes
- 1 hour and 35 minutes of high-definition video.
- Expert-led instruction and exploration of important concepts surrounding Azure Web Apps.
What You Will Learn
- How to deploy and configure Azure web apps.
- How to implement Azure Web Jobs.
- Scaling and resilience with Azure web apps.
Hello and welcome back. In this section, we'll discuss how to configure diagnostics, monitoring and analytics.
Running a web app without suitable monitoring and diagnostics means you miss out on a wealth of information about your user's experience. Are users experiencing any errors? Is the web app responsive? What is working and what is not? Which features are being used? Without diagnostics, monitoring and analytics, you are, so to speak, flying blind.
This section will describe the diagnostics, analytics and monitoring tools available for an Azure-hosted web app. We will discuss logging, diagnostic configuration, remote debugging and endpoint monitoring. We'll also discuss the creation and configuration of alerts that can help actively notify you of any potential or current problems. Lastly, we'll cover the options for monitoring web app resource usage to ensure that your application is running optimally.
Firstly, let's review the available log types that we'll be dealing with. An Azure host web app can leverage several types of logs, each specialized for presenting data for a specific source in a specific format. This includes the event log, web service logs, detailed error message logs, failed request tracing logs, as well as application diagnostic logs.
Let's take a look at each of these log types. The event log is the equivalent of the Windows Event Log on Windows machines. A single XML file per web app is used to store events. The event log is useful for capturing any unhandled exceptions that may have escaped the application's error handling logic and we're handled by the system. Web server logs are simple text files with an entry for each arriving HTTP request. Detailed error message logs are HTML files generated by the server for any failed request where their HTTP response code was 400 or higher. One error message is captured per HTML file. Failed request tracing logs are XML files generated by the server for any failed HTTP request. The file captures not only the error message but also the stack trace that led to the error. These XML files come with an XSL stylesheet for friendly web browser viewing. An XML file is created for each failed request trace. Lastly, we have the application diagnostic logs. These are text based logs created by the web app code itself. Details such as log file location, log name, format and content are specific to the web app and the platform it's built on.
Let's go to Visual Studio and view the logs within the Server Explorer. Viewing logs through Visual Studio can be done through the Server Explorer. You can locate that by clicking View on the top menu and then selecting the third option, which is Server Explorer. When you're in here, you'll have the Azure node, which you can right-click and select Connect to Microsoft Azure Subscription. Once you've done that, you'll be able to have access to all of your app services, and you can expand the node like I just have, and you can come into your log files and find any log files that you have on that system. You can select any of the available log files. It would download the data and open it in Visual Studio. Here, we just have a simple XML file for our log file. The available logs will depend on the enabled diagnostics for your web app.
Site Control Manager, called Kudu, is available with every web deployment. Provided you have admin access to the web app, Kudu provides a set of useful and powerful tools which require only the browser to access. These tools include detailed environment descriptions that provide you of all the environment parameters from CPU count to .NET version, application settings and environment variables. Kudu provides a PowerShell and command line remote execution console, allowing you to run commands directly against your web app server from the browser. Kudu also provides a process explorer, log streaming, web job dashboard and a number of other handy utilities.
Let's access the instance for our web app by going to the appropriate URL, which is your web app's Azure .NET URL with scm appended as the subdomain. Let's access the instance for our web app by going to the appropriate URL, which is your web app's Azure website .NET URL, with scm appended as the subdomain. You'll be automatically logged in if you're already logged into the Azure portal and have admin rights.
Now to access the Site Control Manager or Kudu via our webpage, we need to apply the subdomain of scm. So we've got newappca.azurewebsites. Let's put the scm in between both of those. And now we have access to the Kudu dashboard and we've been logged immediately and because I have access rights as admin and I've already been logged in.
So let's have a look at some of the key features. On the top menu, we've got Environment, Debug console, Process Explorer, Tools and Site extensions. Let's click on the Debug console and have a look at the command prompt that it will provide for us. Here we see the remote command and PowerShell consoles, allowing us to run commands against our web server. So we can do the simple commands like dir, as we would if we were on our Windows machine. This is the typical command prompt we should have on a Windows machine. We're not limited to that, though. We can also access the PowerShell command prompt. And here, all of our commands work just as they would would do on the PowerShell command prompt on our local machine.
We also have the Process Explorer at the top next to the Debug console submenu. Here we can find any processes that may be running on our system. We're able to stop profiling them and view the properties of them like so. Here we can view the relevant worker processes running along our server, along with all the stats that we might require. We won't be going into all the features in this course, but you are encouraged to explore the functionality available.
It's also possible to stream logs through PowerShell. For a simple example, we can use Get-AzureWebsiteLog to stream the web app log directly to our console. For that, we'll need to go to PowerShell. I'm here on the PowerShell command line, and to view the logs for our Azure website, it's a simple matter of typing in the following command. Get-AzureWebsiteLog, and then we have to give it the name flag with the name of the web service that we want to view the logs of, and then we give it the command to tail it. So when we press Enter, it should take a short while to attach to the log and then we can see live activity as it occurs on the endpoint. So now it's connected and now we just have to wait for data to come in. And as you can see, it's now streamed in loads of web activity. And that's it, a simple way to stream one of the web app logs.
Similarly to the Azure PowerShell cmdlets, the cross-platform command line interface provides a variety of everyday commands they can include downloading, diagnostic logs and viewing streaming logs. But it's also possible to access and download the various log files with an FTP client. Before doing this, deployment credentials must be set up, as an FTP client cannot authenticate in the same way that a user authenticates with the Azure portal. These credentials are the same credentials that can be used for deployments, meaning that they are the same credentials that would be used by Git client when performing a deployment. Once these credentials are setup, you can use an FTP client of your choosing to connect to the server, browse the available directories and download the required data.
Let's see this in action now. I'm here on the Azure portal and we're gonna check logs via the FTP protocol. Firstly, we'll need to set the login credentials, and you can do that by going down to the option below the deployment source which says deployment credentials underneath publishing. And we'll rename that to CAUserTest, and then we'll give it a suitable password. Make sure that they match. And then you can click Save. And when that's saved, we can go down to the diagnostics log setting under features on the Settings menu. And if you copy the link for the FTP setting half way down the logs panel and then open up Windows Explorer, paste what you've copied and press Enter, it will then ask you for your username and password.
It's important to note when you're typing in your username, you need to use the full username, which you can find on the main panel here where it says FTP/Deployment username. So NewAppCA\CAUserTest. Let's copy that and we can paste that in as the username, password that you need in the password section. Click Log On and the log files are located in this folder. You can access the application log, HTTP log, all of the logs that you'll need in order to diagnose any problems or any issues you have with your web service. That concludes the demo for accessing your log files via FTP.
Diagnostics can be configured through the portal, PowerShell or the cross-platform client. The options available allow us to adjust logging levels, turn various logging features on or off, define where logs are stored and how long they should be stored for, among other things. It's worth noting that the Event Log is always enabled and cannot be disabled.
Let's take a look at how we can configure the logging levels and switch them on and off within the Azure portal now. I'm here in the Azure portal and if we click on all settings for this app, which is the app we've been using so far, and if we go down to the diagnostic logs, here we can see a variety of options we discussed, application logging, web server logging, detailed error messages, as well as failed request tracing.
As a note, when using blob storage, the new portal does not allow us to set the retention periods at the time of writing. For that, we'll need to use the classic portal, which you can access by opening a new browser and typing manage.windowsazure.com. And if we click on our new app, then go to Configure on the menu at the top. Here is where we'll able to set the retention period for the logs in storage. If we set the retention, it will ask us how many days do we want to save those logs for. This is a significant feature as storage is not free, and having logs building up over long period of times may not be desirable and can be quite costly. So just be aware of this feature.
Identifying a problem with a remote web app can be a difficult task. However, it can be made easier with remote debugging. Remote debugging allows us to attach a local copy of Visual Studio to the remote web app. This allows us to step through the code, set breakpoints and inspect values and properties of the running application. When selecting the Azure hosted web app to attach to, it may take up to 30 seconds to attach the first time as it automatically configures the web app for debugging.
As a word of caution though, if you pause a remote debugging session, your web app will not respond to other requests until you resume again. It is therefore not a good idea to rely on this method of debugging for production applications, as it will impact all users and make them think the web app is down. Furthermore, if you pause the web application for more than a few minutes, Azure will assume your web app work process has become unresponsive and will attempt to restart it.
Here we see Visual Studio's Server Explorer showing the available app services. Right clicking on an individual web app provides a context menu with one of the options being Attached Debugger. This is all that's required to start a remote debugging session.
Endpoint monitoring is the concept of having remote processes query your web app to determine its status. These remote processes could be situated around the world, possibly in multiple different locations. As an example, you can configure a process in a European data center to send a request to a URL you define, being the endpoint as a defined interval. The result of this query, whether it be a successful response, a timeout or an error, is captured and can be used to populate monitoring data on the status dashboard for your web app. Based on the captured data, alerts can be configured to be sent when custom thresholds are breached. For example, when a request to your web app times out three times in a row or returns an error.
Let's see in the classic Azure portal how we can configure some endpoint monitoring. Once the classic portal has been loaded and you've navigated to your web app, you can see on the dashboard page web endpoint status, just underneath the graph that appears. If we click configure endpoint monitoring, it will take you to the configure page. And if you scroll down, part way down to the page until you can find the endpoint monitoring section, under monitoring, here you can configure endpoints to be monitored or you just give it a name, it will put in the web address for your web service. And then you can specify which location, various points around the world, Chicago, Singapore, Ashburn, for example. We'll select one of these, and that'll be Stockholm. Based on the captured data, as we've said before, alerts are configurable. And when custom thresholds have been breached, we can return an error if the web app has timed out three times in a row, for example. This is how you can configure endpoints in the classic Azure portal.
Alerts are a feature that allow us to send a notification email whenever a specific threshold is breached. We specify the thresholds by choosing a metric, condition and threshold value and unit. We then specify the evaluation criteria for the alert, also known as a period such as average over the last 15 minutes. Having specified the various criteria, we then specify the email recipients.
Let's have a look at an example. We need to navigate to our app service plan and then alerts Blade. I'm here back on the Azure portal for our application. We wanna go to the App Service plan under the Settings menu. Now we're gonna click on the Alerts setting. We have no alerts set at the moment, so what we can do is we're gonna click Add alert at the top of that panel. So let's create an alert to monitor the CPU usage and alert us when the CPU usage exceeds 80%. We've got our resource selected for us already, so we're gonna say CpuAlert, CPU THRESHOLD BREACHED. It's gonna be a metrics alert, and we're gonna say that the condition is gonna be greater than. And of course, it's gonna be set to 80 for 80% over the last five minutes. Let's go ahead and choose to email the owners, contributors and readers. And let's just click OK. And that's it, we've created our first alert.
Azure provides several tools to help you monitor the status, health and resource consumption of your web app. Charts can be configured to provide a view of the resources in use, such as CPU time, memory utilization, data transfer, number of requests and endpoint uptime, among others. The portal can also help you compare usage against the resources available on the current app service plan. This helps assess whether the current plan is adequate and whether the application is underutilizing resources or hitting a ceiling that may impact its functionality.
The quota usage shown depends on the app service plan in use. For example, a standard plan will only show the storage quota as that is the only relevant quote to show. On a free plan, the quota summary will include a variety of items such as CPU usage, data in, data out and memory usage. The portal also allows us to view the audit log when required. As a note, we'll be making use of the new portal, as well as the classic portal for this demonstration due to existing differences in functionality.
I'm here on the Azure portal and I've got the dashboard page for our app showing and we can see a rather useful selection of reports that we can toggle on or off by clicking on these toggles at the top here. CPU time, data in, data out, HTTP server errors and so on. We can also see a usage overview at the bottom here. This one only covers the file system. There's no more points that we can monitor at this stage for an app on this service plan. The free plan, for example, will show a greater variety of usage data points as it's far more limited than the standard plan, so it will show a few more items here on this page. For a free plan, you might see CPU time, memory, as well as data in and out.
Let me go and demonstrate how that would appear. If we go to Scale and we'll scale down to the free app service plan, click Save and see what that looks like on the dashboard page when that has taken effect. Now that it's taken effect, let's go to the dashboard page and you can see that there's more usage overview data points here for you to view. Let's go back and revert that to the standard plan and click Save. And let's go back to the new portal and we can see the web app and activity logs there. And if we select under Settings Activity logs, now we've got the new portal available.
Now with these logs, there's another useful feature when analyzing web app resources. You've got loads of filters here that you can apply in order to narrow down and drill down more into the data that's available.
In this section, we covered the topics of viewing and retrieving diagnostic data such as logs. We set the various log types and the ways to access them. We discussed the concept of remote debugging using Visual Studio and how to debug live application. We covered endpoint monitoring, a key monitoring tool used to report on the availability and health of your web app from remote locations around the globe. We discussed alerts and alert configuration to help notify us about various conditions in our web app environment. Lastly, we covered the topic of resource monitoring to provide insight to web app environment health and capacity.
Next we will discuss web jobs and how they can help us deal with the various workloads.
About the Author
Isaac has been using Microsoft Azure for several years now, working across the various aspects of the service for a variety of customers and systems. He’s a Microsoft MVP and a Microsoft Azure Insider, as well as a proponent of functional programming, in particular F#. As a software developer by trade, he’s a big fan of platform services that allow developers to focus on delivering business value.