Firehose Delivery Stream

Contents

keyboard_tab
Course Introduction
1
Introduction
PREVIEW41s
Kinesis Analytics
Course Conclusion
play-arrow
Start course
Overview
DifficultyIntermediate
Duration25m
Students1889
Ratings
3.5/5
starstarstarstar-halfstar-border

Description

This course will provide you with a good foundation to better understand Amazon Kinesis, along with helping you to get started with building streamed solutions. In this course, we'll put a heavier emphasis on hands-on demos along with breaking down the concepts and introducing you to the components that make up Amazon Kinesis.

Intended audience

  • People working with Big Data
  • Business intelligence
  • DevOps
  • Development

Learning Objectives

  • Demonstrate knowledge of Amazon Kinesis, what are the core aspects of the service (streams, firehose, analytics), their common use cases, big data use cases, and how do you access / integrate with the Kinesis service Demonstrate how to work with and create Kinesis streams
  • Demonstrate how to work with Kinesis Firehose and how to use Firehose with Redshift
  • Set up monitoring and analysis of the stream services
  • Understand how to extract analysis from Kinesis

This Course Includes

  • 45 minutes of high-definition video
  • Live demonstrations on key course concepts

What You'll Learn

  • What is Streaming Data: An overview of streaming data and it’s common uses.
  • Setting Up Kinesis Agent: In this demo, we're working with installing the Amazon Kinesis Stream agent.
  • Kinesis Streams: An overview of Kinesis Streams, what they do, and common use cases.
  • Performing Basic Stream Operations: In this demo, we'll be pulling a basic Amazon Kinesis stream from the command line.
  • Firehose: In this lesson, we'll be discussing the fully managed solution, Amazon Kinesis Firehose.
  • Firehose Delivery Stream: In this demo, we're going to set up an Amazon Kinesis Firehose stream.
  • Testing Delivery Stream: In this lesson, we're going to do a quick follow up to the Firehose stream, and test the data delivery.
  • Kinesis Analytics: In this lesson, we'll go over the analytics components of Kinesis.
  • Kinesis Analytics Demo: In this demo, we're going to begin working with Amazon Kinesis Analytics.
  • Kinesis Features Comparison: In this lesson, we'll compare some products within the Amazon Kinesis suite, as well as some other Amazon services.
  • Course Conclusion: A wrap-up and review of the course.

Updates

28/05/2019 - Re-record of lectures to improve audio 

Transcript

Welcome back to Working with Amazon Kinesis. I'm Richard Augenti and I'll be your instructor for this demo. In this demo we're gonna set up an Amazon Kinesis Firehose stream. So, let's get started. So, let's navigate to and go to Firehouse console and we're gonna create a delivery stream. Great. So, we're gonna select the Amazon S3 as our destination and we're gonna provide a name for our stream, which in our case we're gonna do StreamDemoS3, and we're gonna select the bucket, and we're gonna create a new bucket here.

And we're gonna select our region and create bucket. And it'll set up it up to be all lowercase. Okay, great. We're gonna create our bucket. Awesome. Let's click next. We're not gonna enable data transformations. I'm not gonna need to transfer any logs to any kind of different formats. Buffer size is fine, buffer interval's fine. We could set it up to do different data compression, so there's no need to test them all and of course we do data encryption. We have error logging enabled and here's the location where we can actually set up our Firehose role for authentication so that we can access our S3 bucket and other AWS services.

Okay, great, so I'm gonna allow our role. So, I'm gonna click on next and I've just gotta verify the information, make sure everything's correct and then we create our stream. Awesome. So, now we've created our Amazon Kinesis Firehose stream. Now let's check it out. Our settings there in place. And this will be where we're able to monitor and once everything's started flowing into our stream. Great. So, that wraps up our demo on creating an Amazon Kinesis Firehose. I'm Richard Augenti and I'll see you in the next lesson.

About the Author
Students322
Courses1
Learning paths1

Richard Augenti is a DevOps Engineer with 23 years of IT professional experience and 7 years of cloud experience with AWS and Azure. He has been engaged with varying sized projects with clients all across the globe including most sectors. He enjoys finding the best and most efficient way to make things work so, working with automation, cloud technologies, and DevOps has been the perfect fit. When Richard is not engaged with work, he can also be found presenting workshops and talks at user conferences on cloud technologies and other techie talks.