Combining and Enriching Data with Amazon Managed Workflows for Apache Airflow

Amazon Managed Workflows for Apache Airflow (MWAA) is a secure and highly available workflow orchestration tool. Using Amazon MWAA saves you from the technical complexity of creating, managing, and configuring the servers and other resources required by an Apache Airflow environment.

Learning how to create and use Amazon MWAA will make you more effective at creating and working with data processing systems in the AWS public cloud.

In this hands-on lab, you will tour the Amazon MWAA environment creation options, create a data source for use with Amazon MWAA, and you will create a workflow and run it in Amazon MWAA.

Please note: This lab creates a new Amazon MWAA environment for you from scratch. This process can take up to 30 minutes to complete. You should have at least this amount of time available before starting this lab. You can begin the lab and complete the first two lab steps before lab setup has fully completed.

Learning Objectives

Upon completion of this intermediate level lab, you will be able to:

• Understand the requirements and options for a new Amazon MWAA environment
• Update and configure an AWS Lambda function
• Use an IDE to create a directed acyclic graph (DAG) in Python
• Use Apache Airflow to run your DAG

Intended Audience

• Candidates for AWS certification
• Cloud Architects
• Data Engineers
• DevOps Engineers
• Machine Learning Engineers
• Software Engineers

Prerequisites

Familiarity with the following will be beneficial but is not required:

• Amazon Managed Workflows for Apache Airflow (MWAA)
• AWS Lambda
• The Python scripting language
• Amazon Simple Storage Service (S3)

The following content can be used to fulfill the prerequisites:

• Amazon Managed Workflows for Apache Airflow
• Use AWS Lambda Custom Runtimes to Run Bash Commands
• Python for Beginners
• Introduction to Amazon S3

Updates

July 18th, 2023 - Resolved deployment issue

April 14th, 2023 - Enabled autosave for theia container