Processing Streaming Metadata using Amazon Kinesis Data Streams
Amazon Kinesis Data Streams (KDS) is a serverless data streaming service available in the public AWS cloud. It's a fully managed service that can be used to store and ingest streaming data in real time at scale. Amazon KDS integrates with other AWS services as well as being easy to use from custom applications.
Learning how to set up, configure, and use Amazon KDS will enable you to build systems in the AWS cloud that can handle high throughput data rates.
In this hands-on lab, you will create a new Amazon Kinesis data stream and configure an AWS Lambda function to process data records from the stream.
Learning Objectives
Upon completion of this beginner level lab, you will be able to:
- Create a new Amazon Kinesis Data Stream
- Configure an AWS Lambda function to process data stream records
- Configure an Amazon Simple Queue Service (SQS) queue to handle processing failures
Intended Audience
- Candidates for the AWS Certified Data Analytics Specialty certification
- Cloud Architects
- Data Engineers
- DevOps Engineers
- Software Engineers
Prerequisites
Familiarity with the following will be beneficial but is not required:
- Amazon Kinesis Data Streams (KDS)
- AWS Lambda
- Amazon Simple Queue Service (SQS)
The following content can be used to fulfill the prerequisites:
- Amazon Kinesis Data Streams Part 1
- Amazon Kinesis Data Streams Part 2
- Understanding AWS Lambda to Run & Scale Your Code
- Decoupled and Event-Driven Architectures in AWS
Updates
February 22nd, 2024 - Resolved deployment issue
October 25th, 2023 - Updated the instructions and screenshots to reflect the latest UI
Andrew is a Labs Developer with previous experience in the Internet Service Provider, Audio Streaming, and CryptoCurrency industries. He has also been a DevOps Engineer and enjoys working with CI/CD and Kubernetes.
He holds multiple AWS certifications including Solutions Architect Associate and Professional.