Designing Data Flows in Azure
Data Flow Basics
Designing a Data Flow Solution
The course is part of this learning path
This Designing Data Flows in Azure course will enable you to implement the best practices for data flows in your own team. Starting from the basics, you will learn how data flows work from beginning to end. Though we do recommend an idea of what data flows are and how they are used, this course contains some demonstration lectures to really make sure you have got to grips with the concept. By better understanding the key components available in Azure to design and deploy efficient data flows, you will be allowing your organization to reap the benefits.
This course is made up of 19 comprehensive lectures including an overview, demonstrations, and a conclusion.
- Review the features, concepts, and requirements that are necessary for designing data flows
- Learn the basic principles of data flows and common data flow scenarios
- Understand how to implement data flows within Microsoft Azure
- IT professionals who are interested in obtaining an Azure certification
- Those looking to implement data flows within their organizations
- A basic understanding of data flows and their uses
Related Training Content
For more training content related to this course, visit our dedicated MS Azure Content Training Library.
When designing a data flow, storage is a key piece of the flow. As such, it's important to consider what options are available. In this lesson, we're going to look at Azure Storage, the blob storage offering in particular. Because blob storage offers multiple tiers, including premium, hot, cool, and archive, data storage costs will vary widely depending on the tiers you need or use. Obviously, data stored and accessed in the premium tier is going to result in the best performance and lowest latencies. Of course, it's also going to the the most expensive. The Hot tier offers good performance for data that will be accessed frequently, but at a lower cost than premium. For infrequently accessed data that still needs to be available with no delay, the Cool tier is a good fit. The Archive tier, however, should be reserved for data that needs to be kept but, when needed, isn't needed immediately. As you move down from premium, to hot, to cool, and then to archive, data storage and access costs are going adjusted to reflect the intent of the particular storage tier. The premium and hot tiers charge more for access, since the data needs to be at the ready, while the cool and archive tiers charge less for access, since the data is less readily available than the other tiers. Blobs will ultimately be the cheapest storage option available to you for storing data as part of your data flow.
You essentially pay for storage only, no compute. Your storage costs will be determined by how much data you're storing, and in which tier you are storing it. A single blob can host upwards of five terabytes, and a storage account can host up to two petabytes, they're a great choice for housing large amounts of data. This makes them perfect for initial data ingestion prior to processing it. And because blobs are built for unstructured data, they are great in cases where ELT is involved because you can really store just about anything in them, including media, video, images, and even CSV files. Blog storage is also a good choice for keeping RAW data around long term due to its cost. As such, you can leverage it to cheaply store your raw data to re-process it later, if and when new questions about the business come up.
About the Author
Tom is a 25+ year veteran of the IT industry, having worked in environments as large as 40k seats and as small as 50 seats. Throughout the course of a long an interesting career, he has built an in-depth skillset that spans numerous IT disciplines. Tom has designed and architected small, large, and global IT solutions.
In addition to the Cloud Platform and Infrastructure MCSE certification, Tom also carries several other Microsoft certifications. His ability to see things from a strategic perspective allows Tom to architect solutions that closely align with business needs.
In his spare time, Tom enjoys camping, fishing, and playing poker.