Designing Data Flows in Azure
Data Flow Basics
Designing a Data Flow Solution
The course is part of these learning pathsSee 3 more
This Designing Data Flows in Azure course will enable you to implement the best practices for data flows in your own team. Starting from the basics, you will learn how data flows work from beginning to end. Though we do recommend an idea of what data flows are and how they are used, this course contains some demonstration lectures to really make sure you have got to grips with the concept. By better understanding the key components available in Azure to design and deploy efficient data flows, you will be allowing your organization to reap the benefits.
This course is made up of 19 comprehensive lectures including an overview, demonstrations, and a conclusion.
- Review the features, concepts, and requirements that are necessary for designing data flows
- Learn the basic principles of data flows and common data flow scenarios
- Understand how to implement data flows within Microsoft Azure
- IT professionals who are interested in obtaining an Azure certification
- Those looking to implement data flows within their organizations
- A basic understanding of data flows and their uses
Related Training Content
For more training content related to this course, visit our dedicated MS Azure Content Training Library.
Azure Synapse is a cloud-based analytics service in Azure that combines enterprise data warehousing and Big Data analytics. This service allows you to query data using either serverless on-demand or provisioned resources at scale. At a high level, what Azure Synapse does it allow you to ingest, prepare, manage, and serve data for immediate business intelligence and machine learning needs.
The Azure Synapse offering consists of 4 parts. They include Synapse SQL, Spark, Synapse Pipelines, and Studio. Synapse SQL provides complete T-SQL based analytics via a dedicated SQL pool or a serverless SQL pool, while Spark is a deeply integrated Apache Spark implementation. Synapse Pipelines provides hybrid data integration, while Studio offers a unified user experience. I should mention, however, that the serverless SQL pool piece, the integrated Spark implementation, Synapse Pipelines, and Studio are all currently in preview at the time of this lesson recording.
The Dedicated SQL pool in Synapse Analytics is really a collection of analytic resources that are provisioned when you use Synapse SQL, and the size of the pool is determined by Data Warehousing Units, or DWUs.
Azure Synapse allows you to import big data, using PolyBase T-SQL queries. Its distributed query engine will then allow you to run high-performance analytics on that data.
Because data warehousing is a huge piece of any cloud-based big data solution, Azure Synapse plays a key role.
For example, in a typical cloud-based big data solution, the data is first ingested into big data stores from various sources. Once the data makes its way into the big data store, it’s prepared and trained by a solution like Hadoop, Spark, and machine learning algorithms.
After being prepared and trained, the data is then ready for complex analysis. At this point, the dedicated SQL pool queries the big data stores, using PolyBase, which, in turn, uses standard T-SQL queries to pull the data into dedicated SQL pool tables.
The Dedicated SQL pool stores the data in relational tables with columnar storage. Storing the data in this fashion reduces data storage costs, while also improving query performance.
You can run analytics on the stored data at a massive scale. Such analysis queries will often complete in seconds, instead of minutes, when compared to traditional database systems.
Results of the analysis can be sent to reporting databases or applications all over the world, where business analysts will use the information to make better-informed business decisions.
If you are interested in reading about the nuts and bolts of the architecture of Azure Synapse Analytics, visit the URL that you see on your screen
Tom is a 25+ year veteran of the IT industry, having worked in environments as large as 40k seats and as small as 50 seats. Throughout the course of a long an interesting career, he has built an in-depth skillset that spans numerous IT disciplines. Tom has designed and architected small, large, and global IT solutions.
In addition to the Cloud Platform and Infrastructure MCSE certification, Tom also carries several other Microsoft certifications. His ability to see things from a strategic perspective allows Tom to architect solutions that closely align with business needs.
In his spare time, Tom enjoys camping, fishing, and playing poker.