hands-on lab

Combining and Enriching Data with Amazon Managed Workflows for Apache Airflow

Intermediate
Up to 2h
332
5/5
Get guided in a real environmentPractice with a step-by-step scenario in a real, provisioned environment.
Learn and validateUse validations to check your solutions every step of the way.
See resultsTrack your knowledge and monitor your progress.

Description

Amazon Managed Workflows for Apache Airflow (MWAA) is a secure and highly available workflow orchestration tool. Using Amazon MWAA saves you from the technical complexity of creating, managing, and configuring the servers and other resources required by an Apache Airflow environment.

Learning how to create and use Amazon MWAA will make you more effective at creating and working with data processing systems in the AWS public cloud.

In this hands-on lab, you will tour the Amazon MWAA environment creation options, create a data source for use with Amazon MWAA, and you will create a workflow and run it in Amazon MWAA.

Please note: This lab creates a new Amazon MWAA environment for you from scratch. This process can take up to 30 minutes to complete. You should have at least this amount of time available before starting this lab. You can begin the lab and complete the first two lab steps before lab setup has fully completed.

Learning Objectives

Upon completion of this intermediate level lab, you will be able to:

  • Understand the requirements and options for a new Amazon MWAA environment
  • Update and configure an AWS Lambda function
  • Use an IDE to create a directed acyclic graph (DAG) in Python
  • Use Apache Airflow to run your DAG

Intended Audience

  • Candidates for AWS certification
  • Cloud Architects
  • Data Engineers
  • DevOps Engineers
  • Machine Learning Engineers
  • Software Engineers

Prerequisites

Familiarity with the following will be beneficial but is not required:

  • Amazon Managed Workflows for Apache Airflow (MWAA)
  • AWS Lambda
  • The Python scripting language
  • Amazon Simple Storage Service (S3)

The following content can be used to fulfill the prerequisites:

Updates

July 18th, 2023 - Resolved deployment issue

April 14th, 2023 - Enabled autosave for theia container

Environment before

Environment after

About the author

Students
68,200
Labs
168
Courses
2
Learning paths
4

Andrew is a Labs Developer with previous experience in the Internet Service Provider, Audio Streaming, and CryptoCurrency industries. He has also been a DevOps Engineer and enjoys working with CI/CD and Kubernetes.

He holds multiple AWS certifications including Solutions Architect Associate and Professional.

Covered topics

Lab steps

Logging In to the Amazon Web Services Console
Touring the Amazon Managed Workflows for Apache Airflow Console
Creating a Data Source
Creating a Directed Acyclic Graph
Running your Directed Acyclic Graph