Combining and Enriching Data with Amazon Managed Workflows for Apache Airflow
Amazon Managed Workflows for Apache Airflow (MWAA) is a secure and highly available workflow orchestration tool. Using Amazon MWAA saves you from the technical complexity of creating, managing, and configuring the servers and other resources required by an Apache Airflow environment.
Learning how to create and use Amazon MWAA will make you more effective at creating and working with data processing systems in the AWS public cloud.
In this hands-on lab, you will tour the Amazon MWAA environment creation options, create a data source for use with Amazon MWAA, and you will create a workflow and run it in Amazon MWAA.
Please note: This lab creates a new Amazon MWAA environment for you from scratch. This process can take up to 30 minutes to complete. You should have at least this amount of time available before starting this lab. You can begin the lab and complete the first two lab steps before lab setup has fully completed.
Upon completion of this intermediate level lab, you will be able to:
- Understand the requirements and options for a new Amazon MWAA environment
- Update and configure an AWS Lambda function
- Use an IDE to create a directed acyclic graph (DAG) in Python
- Use Apache Airflow to run your DAG
- Candidates for AWS certification
- Cloud Architects
- Data Engineers
- DevOps Engineers
- Machine Learning Engineers
- Software Engineers
Familiarity with the following will be beneficial but is not required:
- Amazon Managed Workflows for Apache Airflow (MWAA)
- AWS Lambda
- The Python scripting language
- Amazon Simple Storage Service (S3)
The following content can be used to fulfill the prerequisites:
- Amazon Managed Workflows for Apache Airflow
- Use AWS Lambda Custom Runtimes to Run Bash Commands
- Python for Beginners
- Introduction to Amazon S3
July 18th, 2023 - Resolved deployment issue
April 14th, 2023 - Enabled autosave for theia container
Andrew is a Labs Developer with previous experience in the Internet Service Provider, Audio Streaming, and CryptoCurrency industries. He has also been a DevOps Engineer and enjoys working with CI/CD and Kubernetes.
He holds multiple AWS certifications including Solutions Architect Associate and Professional.