Azure Data Factory (ADF) is a managed cloud service for ingesting, preparing and transforming data from multiple sources. ADF provides code-free, visual data pipeline interface to describe workflows allowing data engineers and non-expert data integrators alike to accomplish complex data manipulation tasks. Over 90 data sources are supported including Azure services, Amazon Web Services, Google Cloud services, Teradata, and Salesforce.
This lab introduces you to Azure Data Factory. You will get acquainted with ADF by performing an Azure Blob Storage data movement operation using an ADF pipeline.
Warning: Currently, Data Factory UI is only officially supported by Microsoft Edge and Google Chrome web browsers. It is also recommended to use incognito mode to avoid conflicts with other Microsoft accounts impacting Data Factory UI.
Learning Objectives
Upon completion of this intermediate-level lab, you will be able to:
- Create an Azure Data Factory
- Understand and initialize ADF linked services
- Initialize ADF datasets
- Develop basic data pipelines in Data Factory UI
- Learn to Debug and Trigger ADF pipelines
- Triggers ADF pipeline runs on a schedule
Intended Audience
- Candidates for Microsoft Azure Data Engineering Certifications
- Data Engineers
Prerequisites
Familiarity with the following will be beneficial but is not required:
- Basic understanding of Azure Data Factory
- Basic understanding of Azure Blob Storage
The following courses can be used to fulfill the prerequisite:
Portions of this lab's content have been adapted from attributed sources.
Updates
October 3rd, 2023 - Resolved deployment issue
Logan has been involved in software development and research since 2007 and has been in the cloud since 2012. He is an AWS Certified DevOps Engineer - Professional, AWS Certified Solutions Architect - Professional, Microsoft Certified Azure Solutions Architect Expert, MCSE: Cloud Platform and Infrastructure, Google Cloud Certified Associate Cloud Engineer, Certified Kubernetes Security Specialist (CKS), Certified Kubernetes Administrator (CKA), Certified Kubernetes Application Developer (CKAD), and Certified OpenStack Administrator (COA). He earned his Ph.D. studying design automation and enjoys all things tech.