Data Flow Basics
Data Flow Components
Building a Dataflow with Azure Data Factory
The course is part of these learning pathsSee 3 more
In this course, we're going to review the features, concepts, and requirements that are necessary for designing data flows and how to implement them in Microsoft Azure. We’re also going to cover the basics of data flows, common data flow scenarios, and what all is involved in designing a typical data flow.
- Understand key components that are available in Azure that can be used to design and deploy data flows
- Know how the components fit together
This course is intended for IT professionals who are interested in earning Azure certification and for those who need to work with data flows in Azure.
To get the most from this course, you should have at least a basic understanding of data flows and what they are used for.
Welcome to Azure Data Lake Storage. In this brief lesson, I want to touch on Azure Data Lake Storage.
Azure Data Lake Storage Gen2 offers the analytics performance of an HDFS-compatible file system, while ALSO offering tiering and data lifecycle management features of blob storage. Other features and benefits now available in Gen 2, as a result, include data security, data durability, and replication options that you didn’t have before.
With Azure Data Lake Storage Gen 2, you have access to not only the Azure Data Lake Gen2 file system APIs, but also to the Blob REST APIs. As a result, you can interact with data in different ways.
The best part about it is the fact that enabling all this cool functionality is as simple as changing a setting in the storage account. Instead of needing to provision a new service and maintain a new service, and such, all you need to do is enable it on the storage account.
A key feature of Data Lake Storage Gen2 is the hierarchical namespace that’s added to blob storage. With a hierarchical namespace, objects and files are organized into a hierarchy of directories, which results in much more efficient data access. For example, renaming a directory becomes a single metadata operation on the directory being renamed, instead of a process that enumerates and processes all objects that share the name prefix of the directory.
Other key features of Azure Data Lake Storage Gen 2 include Hadoop compatible access and a security model that supports ACL and POSIX permissions, including some extra granularity that’s specific to Data Lake Storage Gen 2. With an ABFS driver that’s optimized for big data analytics, data access is improved. And because Data Lake Storage Gen 2 offers low-cost storage capacity, as well as low-cost transactions, the service is cost-effective.
Tom is a 25+ year veteran of the IT industry, having worked in environments as large as 40k seats and as small as 50 seats. Throughout the course of a long an interesting career, he has built an in-depth skillset that spans numerous IT disciplines. Tom has designed and architected small, large, and global IT solutions.
In addition to the Cloud Platform and Infrastructure MCSE certification, Tom also carries several other Microsoft certifications. His ability to see things from a strategic perspective allows Tom to architect solutions that closely align with business needs.
In his spare time, Tom enjoys camping, fishing, and playing poker.