Using Azure Data Factory Pipelines to Copy Data

Lab Steps

lock
Logging in to the Microsoft Azure Portal
lock
Creating an Azure Data Factory
lock
Creating a linked service
lock
Creating the Data Factory Datasets
lock
Creating a Data Factory Pipeline
lock
Debugging the Data Factory Pipeline
lock
Triggering and Monitoring the Pipeline
lock
Triggering the Pipeline on a Schedule

Ready for the real environment experience?

DifficultyIntermediate
Time Limit1h 15m
Students799
Ratings
4.5/5
starstarstarstarstar-half

Description

Azure Data Factory (ADF) is a managed cloud service for ingesting, preparing and transforming data from multiple sources. ADF provides code-free, visual data pipeline interface to describe workflows allowing data engineers and non-expert data integrators alike to accomplish complex data manipulation tasks. Over 90 data sources are supported including Azure services, Amazon Web Services, Google Cloud services, Teradata, and Salesforce.

This lab introduces you to Azure Data Factory. You will get acquainted with ADF by performing an Azure Blob Storage data movement operation using an ADF pipeline.

Warning: Currently, Data Factory UI is only officially supported by Microsoft Edge and Google Chrome web browsers. It is also recommended to use incognito mode to avoid conflicts with other Microsoft accounts impacting Data Factory UI.

Learning Objectives

Upon completion of this intermediate-level lab, you will be able to:

  • Create an Azure Data Factory
  • Understand and initialize ADF linked services
  • Initialize ADF datasets
  • Develop basic data pipelines in Data Factory UI
  • Learn to Debug and Trigger ADF pipelines
  • Triggers ADF pipeline runs on a schedule

Intended Audience

  • Candidates for Microsoft Azure Data Engineering Certifications
  • Data Engineers

Prerequisites

Familiarity with the following will be beneficial but is not required:

  • Basic understanding of Azure Data Factory
  • Basic understanding of Azure Blob Storage

The following courses can be used to fulfill the prerequisite:

Portions of this lab's content have been adapted from attributed sources.

Environment before
PREVIEW
arrow_forward
Environment after
PREVIEW
About the Author
Students112840
Labs181
Courses9
Learning paths30

Logan has been involved in software development and research since 2007 and has been in the cloud since 2012. He is an AWS Certified DevOps Engineer - Professional, AWS Certified Solutions Architect - Professional, Microsoft Certified Azure Solutions Architect Expert, MCSE: Cloud Platform and Infrastructure, Google Cloud Certified Associate Cloud Engineer, Certified Kubernetes Security Specialist (CKS), Certified Kubernetes Administrator (CKA), Certified Kubernetes Application Developer (CKAD), and Certified OpenStack Administrator (COA). He earned his Ph.D. studying design automation and enjoys all things tech.