Azure Storage is a managed cloud storage offering from Microsoft. It provides a scalable store for data objects, a cloud-based file system, and even provides a messaging and NoSQL store. As a managed service, maintenance and support is handled by Microsoft Azure, taking the care and feeding aspect of storage support out of the hands of the Azure administrator.
Durable and highly available, Azure Storage offers redundancy to ensure that data is protected during hardware failures. The ability to replicate data across Azure datacenters and even across different Azure regions offers additional protection against catastrophes and natural disasters. Data that is stored and replicated in Azure remains highly available in the event of unexpected outages.
Massively scalable, Azure Storage can meet the most demanding data storage requirements for today’s applications, including use cases such as big data. In addition to offering such scalable storage, the Azure Storage service also provides secure data storage by encrypting all data that is written to the service. By offering fine-grained control over access to the data, Azure Storage also makes it easy to control such access.
Data housed in Azure Storage is essentially accessible from everywhere in the world via HTTP or HTTPS. However, Microsoft also offers several SDKs for Azure Storage, in several languages, including .NET, Java, Node.js, and many others. Microsoft also offers a mature REST API and scripting abilities in Azure PowerShell and in Azure CLI. GUI access is also provided via the Azure portal and by Azure Storage Explorer.
Azure Storage can essentially be broken down into five key data services, including Azure Blobs, Azure Files, Azure Queues, Azure Tables, and Disk Storage. Azure Blobs offers a scalable object store for storing text and binary data, while Azure Files offers managed file shares that can be used for cloud and on-premises deployments. Azure Queues is a messaging store that is used to facilitate reliable messaging among application components. Azure Tables is a NoSQL store used for “schemaless” storage of structured data. Disk Storage offers unmanaged and managed disks that are used by virtual machines.
All storage services can be accessed through an Azure Storage Account.
Microsoft’s Azure Blob storage offering is intended for object storage in the cloud. As such, it is optimized for storage of unstructured data, such as binary data and text.
Blob storage in Azure is a good choice for serving data such as images for a website directly to a browser and for storing files that are accessed in in a distributed fashion. It also works well for video and audio streaming. Other uses for blob storage include backup data storage, disaster recovery, and data archives.
Objects that are stored in blob storage are accessible worldwide via HTTP or HTTPs. Such blobs are accessed by users and client applications via URLs, Azure PowerShell, the Azure CLI, the Azure Storage REST API, or by an Azure Storage client library, which are available for many languages, such as .NET, Java, Node.js, Python, PHP, and Ruby.
The Azure Files service offers the ability to configure cloud-based file shares that are highly-available and can be accessed via the well-known Server Message Block (SMB) protocol. Azure Files provides the ability to allow multiple virtual machines to share the same files, while providing both read and write access to those files. In addition to accessing them via SMB, the files in an Azure file share can also be accessed using the REST interface and storage client libraries.
What makes Azure Files advantageous over a corporate file share is the fact that Azure Files are accessible from anywhere in the world via a URL and a shared access signature (SAS) token. SAS tokens can be generated to allow specific access to files for a specific amount of time, adding a level of control not available with corporate file shares on-premises.
Because many on-prem applications use traditional file shares for storing data, moving such applications to Azure is typically straightforward. Mounting an Azure-based file share using the same drive letter that the on-prem application uses allows the application to continue accessing the required data (now in the cloud) without any major changes.
Another common use case for file shares hosted in Azure Files is the sharing of crash dumps, diagnostic logs, and the like. In such use cases, applications (both on-prem and in the cloud) can be configured to store log files and such in Azure-based file shares and those logs can be viewed and analyzed later.
At the time of this writing, Microsoft is in the process of rolling out support for Azure Active Directory authentication for Azure Files SMB access. Such integration is a feature that’s been requested for quite some time. This new feature allows access to Azure file shares via SMB from machines joined to an Azure AD Domain Services domain. Such access can then be controlled with Azure AD credentials. Additionally, Azure Files now supports NTFS ACLs on files and folders stored in file shares hosted by Azure Files. These new features allow organizations to sync their on-prem AD accounts to Azure AD and then to access shares that have been migrated from on-prem to Azure, using the same credentials that are used on-prem.
Another feature of Azure Storage is the Azure Queue service. This feature of Azure Storage stores and retrieves messages. Queue messages that are stored by the Azure Queue service can be up to 64 KB in size. An Azure Queue, which is typically used to store lists of messages, can contain millions of messages. Messages stored in Azure Queue are then processed asynchronously.
An example use case for Queue Storage would be a website that produces thumbnails for pictures uploaded by users. Instead of making a user wait while thumbnails are created, a queue could be used. Once the initial upload of pictures is complete, a message can be written to the queue and an Azure Function leveraged to retrieve the message and then create the thumbnails. Such a process could then more easily be scaled.
The original Azure Table storage service provides the ability to store structured NoSQL data in Azure in a schemaless fashion. Since it’s schemaless, data in Table Storage is easy to adapt to the needs of your application. Access to Table Storage is generally more cost effective than traditional SQL.
Table Storage offers the ability to store terabytes of structured data and storing datasets not requiring complex joins, stored procedures, and such can be denormalized for fast access. It also provides the ability to quickly query data via a clustered index, accessing such data using the OData protocol and LINQ queries, with WCF Data Service .NET Libraries.
The Table Storage feature can be used to store and query large data sets that consist of structured, non-relational data.
As of this writing, however, Microsoft has begun offering a premium solution for table storage: The Azure Cosmos DB Table API. Key features of this new offering include global distribution, throughput-optimized tables, and automatic secondary indexes. Access to Table storage data is fast and cost-effective for many types of applications and is typically lower in cost than traditional SQL for similar volumes of data.
Microsoft’s Azure Storage offering also features managed and unmanaged disk capabilities that are used by Azure virtual machines. Such capabilities include the Operating System disk, which host the operating system for a virtual machine, the Temporary Disk, which provides short-term storage for apps and processes, and the Data Disk, which is really a VHD that’s attached to a VM and used to store application data and other data that’s needed.
Disk Storage is typically created and managed while managing Azure virtual machines.
Within Microsoft’s storage offering are multiple type of storage accounts, each capable of handling different types of objects. These include the General-Purpose Standard account, the General-Purpose Premium account, and Blob Storage (hot and cool access tiers).
Standard storage accounts are, by far, the most widely used type of storage account in Azure. A General-Purpose Standard account supports all types of data, including blob, file, queue, and table. It should be noted that standard storage accounts store data on magnetic disk.
General-Purpose Premium storage offers high-performance storage for page blobs (typically used for VHD files). While standard accounts use magnetic media for storing data, premium storage accounts use SSD. As such, it’s typically more expensive to host data in a premium account than a standard account. Naturally, Microsoft recommends Premium Storage for all virtual machines hosted in Azure.
Blob Storage accounts are a specialized breed. They are used to store block blobs and append blobs. However, blob storage accounts do not allow storage of page blobs. As such, they do not support storage of VHD files. These types of storage accounts provide two different access tiers Hot and Cool. The Hot tier would typically be used to store data that is accessed regularly, while the Cool tier would be used to store less-accessed data (ie. archives). The access tier can be changed at any time.
Storage costs for data in the Hot tier are higher, but access costs are lower. Storage costs for data in the Cool tier are lower, but access costs are higher.
With numerous account types and services, cloud administrators have numerous choices when it comes to their storage needs. In our next blog, we’ll discuss practical use cases for Azure Storage. No matter where you are in your Azure storage journey, our Azure Training Library can help you get started.
It's Flash Sale time! Get 50% off your first year with Cloud Academy: all access to AWS, Azure, and Cloud…
In this blog post, we're going to answer some questions you might have about the new AWS Certified Data Engineer…
This is my 3rd and final post of this series ‘Navigating the Vocabulary of Gen AI’. If you would like…