image
Scaling Out Parallel Workloads
Start course
Difficulty
Intermediate
Duration
35m
Students
728
Ratings
4.1/5
Description

This course will show practical applications of key Azure features to meet the programming and configuration challenges introduced by long-running tasks.

We'll start with Azure Batches and how you can use them to create large-scale, parallel, and high-performance apps in the Azure cloud. Then we'll go over Azure Queues and how they can add resiliency to your web applications. Next, you'll look at Webhooks and how they can address events in your cloud apps. Finally, we'll show you WebJobs and how they can deal with continuous processing tasks.

By the end of this course, you should be able to understand and apply these four Azure features to solve some of the challenges you face with long-running tasks, especially in high-performance computing applications.

Learning Objectives

  • Create large-scale, parallel, and high-performance apps by using Azure Batches
  • Build resilient apps by using Azure Queues
  • Implement code to address application events by using Azure Webhooks
  • Address continuous processing tasks by using Azure WebJobs

Intended Audience

  • People pursuing the Microsoft AZ-203 certification
  • IT professionals, web developers, DevOps administrators

Prerequisites

  • Basic understanding of cloud concepts
  • Familiarity with web programming
  • Exposure to Azure configuration (Portal, CLI, or PowerShell)

Feedback 
We’d love to get your feedback on this course, so please give it a rating when you’re finished.

Transcript

The Batch API can handle the scaling out of an intrinsically parallel workload such as image rendering on a pool of up to thousands of compute cores. Instead of having to set up a compute cluster or write code to queue and schedule your jobs and move the necessary input and output data, you could automate the scheduling of large compute jobs and scale a pool of compute VMs up and down to run them. You can write client apps or front ends to run the jobs and tasks on demand, on a schedule, or as part of a larger workflow managed by tools such as Azure Data Factory. 

You can also use the Batch API to wrap an existing application so it runs as a service on a pool of compute nodes that Batch manages in the background. 

In summary, the opportunities presented by HPC are: faster processing of large amounts of data, potential cost savings from the low-utilized compute or spare capacity in the cloud, dedicated and specialized apps, for example, 3D rendering and image processing. 

However, the challenges created by HPC include: administrative and coordination problems arising from DIY HPC, a lack of standards as each project is built from the ground up, and finally buffering of the results as they become available.

About the Author
Students
2163
Courses
2

Derrick is a content contributor and trainer for Microsoft cloud technologies like Azure, Office 365 and Dynamics 365. He works across North America and Europe to help companies and organizations with these technology shifts. Before that he has worn many hats but prefers to wear them one at a time.

When he is not night walking during his travels, you can find him on a bicycle path or performing guitar solos to an imaginary audience in his basement.