Getting the Most From Azure Storage
The course is part of these learning pathsSee 7 more
The Azure Storage suite of services form the core foundation of much of the rest of the Azure services ecosystem. Blobs are low-level data primitives that can store any data type and size. Tables provide inexpensive, scalable NoSQL storage of key/value data pairs. Azure queues provide a messaging substrate to asynchronously and reliably connect distinct elements of a distributed system. Azure files provide an SMB-compatible file system for enabling lift-and-shift scenarios of legacy applications that use file shares. Azure disks provide consistent, high-performance storage for virtual machines running in the cloud.
In this Introduction to Azure Storage course you'll learn about the features of these core services, and see demonstrations of their use. Specifically, you will:
- Define the major components of Azure Storage
- Understand the different types of blobs and their intended use
- Learn basic programming APIs for table storage
- Discover how queues are used to pipeline cloud compute node together
- Learn to integrate Azure files with multiple applications
- Understand the tradeoffs between standard/premium storage and unmanaged/managed disks
Now let's investigate Azure queues. Azure queue storage is a message-oriented data store often used to connect software applications in a producer-consumer relationship where one or more applications are writing input data to the queue, while others are reading from the queue and performing downstream work on each message.
This pattern is useful to solve a number of architecture challenges like gracefully handling bursts of request activity or dependent service inaccessibility, avoiding tight coupling of distributed architecture components, and others. Azure queues are accessible for management tasks and data reads and writes using both native HTTP-based REST API, as well as language-specific SDKs and NET, Java, Node. js, and so on.
Messages in Azure queues consist of 64 kilobytes or less of binary or text data. For larger payloads, it's common to store most of the data out of band in, say, Azure blob or table storage, and simply include a URI pointer in the queue message that points to the real data stored elsewhere.
Also note that queue messages have a built in seven day max time to live window, so this period is renewable via an SDK call. A common point of confusion for Azure architects and developers is understanding the difference between Azure storage queues and Azure service bus queues. Let's explore a few key differences.
First, note that storage queues do not support first in, first out semantics. That is, there's no guarantee of message order delivery. Service bus queues do optionally offer this capability, but at some risk of decreased scalability and throughput. Storage queues ensure that a message will be delivered at least one time to its intended recipient.
In some edge cases, the same message may be delivered twice so item potent message handling, that is the characteristic that processing the same message twice has the exact same side effects each time, is critical with storage queues. Service bus queues can optionally support at-most-once or transactional delivery of messages that avoids duplicates again, at some potential cost to scalability and performance.
Storage and service bus queues both support multiple readers from the same queue. Storage queues use a per message invisibility window that eventually expires if a message is not processed, while service bus uses a pessimistic style of queue-based lock to ensure no two readers process the same message.
Storage queues support up to 200 terabytes of storage, while service bus supports either a one gigabyte or 80 gigabyte max, depending on the service tier chosen. Similarly, storage queues support 64 kilobyte message sizes and a seven day time to live, while service bus supports slightly larger messages of 256K or one megabyte, again depending on the service tier chosen, and an unlimited time to live for messages.
Both technologies support poison message handling, that is the ability to detect a message that has repeatedly been read but unsuccessfully processed. For more details comparing storage queues and service bus queues, see the URL on screen. Let's take a closer look at using storage queues for asynchronous message passing between application components in a typical cloud architecture.
About the Author
Josh Lane is a Microsoft Azure MVP and Azure Trainer and Researcher at Cloud Academy. He’s spent almost twenty years architecting and building enterprise software for companies around the world, in industries as diverse as financial services, insurance, energy, education, and telecom. He loves the challenges that come with designing, building, and running software at scale. Away from the keyboard you'll find him crashing his mountain bike, drumming quasi-rythmically, spending time outdoors with his wife and daughters, or drinking good beer with good friends.