Collecting Log Data with Kinesis Agent and Querying with Amazon Athena

Lab Steps

lock
Creating a Kinesis Firehose Delivery Stream
lock
Connecting to the Virtual Machine using EC2 Instance Connect
lock
Installing and Configuring the Kinesis Agent Application
lock
Querying the Log Data using Amazon Athena

The hands-on lab is part of this learning path

Ready for the real environment experience?

DifficultyBeginner
Time Limit1h 30m
Students59
Ratings
5/5
starstarstarstarstar

Description

Amazon Kinesis Agent is an application that continuously monitors files and sends data to a Amazon Kinesis Data Firehose Delivery Stream or a Kinesis Data Stream. The agent handles rotating files, checkpointing, and retrying upon a failure.

The primary use-case for Kinesis agent is log collection, it's designed to deliver your data quickly and reliably, and also integrates with Amazon CloudWatch to enable you to monitor the health of, and failures in, the streaming process.

In this lab, you will install and configure Kinesis Agent, use it to collect log entries, and query the log entries with Amazon Athena.

Learning Objectives

This is a beginner level lab. Upon completion of this lab you will be able to:

  • Create a Kinesis Firehose Delivery Stream to deliver data to an S3 bucket
  • Install and configure the Kinesis Agent on a Linux EC2 instance to send data to your delivery stream
  • Query your log data stored in S3 using Amazon Athena

Intended Audience

  • Candidates for the AWS Data Analytics Speciality Certification
  • Data Engineers
  • Cloud Engineers

Prerequisites

Familiarity with Analytics in AWS and the Linux command-line will be beneficial but is not required.

The following courses can be used to fulfill the prerequisites:

Environment before
PREVIEW
arrow_forward
Environment after
PREVIEW
About the Author
Students15718
Labs27
Courses1

Andrew is a Labs Developer with previous experience in the Internet Service Provider, Audio Streaming, and CryptoCurrency industries. He has also been a DevOps Engineer and enjoys working with CI/CD and Kubernetes. He holds the AWS Certified Developer - Associate certification.