Creating an Amazon Kinesis Stream


Course Introduction
Kinesis Analytics
Course Conclusion
Start course

This course will provide you with a good foundation to better understand Amazon Kinesis, along with helping you to get started with building streamed solutions. In this course, we'll put a heavier emphasis on hands-on demos along with breaking down the concepts and introducing you to the components that make up Amazon Kinesis.

Intended audience

  • People working with Big Data
  • Business intelligence
  • DevOps
  • Development

Learning Objectives

  • Demonstrate knowledge of Amazon Kinesis, what are the core aspects of the service (streams, firehose, analytics), their common use cases, big data use cases, and how do you access / integrate with the Kinesis service Demonstrate how to work with and create Kinesis streams
  • Demonstrate how to work with Kinesis Firehose and how to use Firehose with Redshift
  • Set up monitoring and analysis of the stream services
  • Understand how to extract analysis from Kinesis

This Course Includes

  • 45 minutes of high-definition video
  • Live demonstrations on key course concepts

What You'll Learn

  • What is Streaming Data: An overview of streaming data and it’s common uses.
  • Setting Up Kinesis Agent: In this demo, we're working with installing the Amazon Kinesis Stream agent.
  • Kinesis Streams: An overview of Kinesis Streams, what they do, and common use cases.
  • Performing Basic Stream Operations: In this demo, we'll be pulling a basic Amazon Kinesis stream from the command line.
  • Firehose: In this lesson, we'll be discussing the fully managed solution, Amazon Kinesis Firehose.
  • Firehose Delivery Stream: In this demo, we're going to set up an Amazon Kinesis Firehose stream.
  • Testing Delivery Stream: In this lesson, we're going to do a quick follow up to the Firehose stream, and test the data delivery.
  • Kinesis Analytics: In this lesson, we'll go over the analytics components of Kinesis.
  • Kinesis Analytics Demo: In this demo, we're going to begin working with Amazon Kinesis Analytics.
  • Kinesis Features Comparison: In this lesson, we'll compare some products within the Amazon Kinesis suite, as well as some other Amazon services.
  • Course Conclusion: A wrap-up and review of the course.


28/05/2019 - Re-record of lectures to improve audio 


- [Richard] Welcome back to Working with Amazon Kinesis. I'm Richard Augenti and I'll be your instructor for this demo. In this demo, we'll be pulling a basic Amazon Kinesis stream from the command line. This exercise is highly valuable because it breaks down each component of what comprises the Amazon Kinesis stream. It's still-- But you're gonna wanna ensure you have more shards available for your production stream. Okay great. So now we're gonna run the described stream command. So we check the status of creating our stream. As you noticed, next to stream status it says creating, which means it's still in the process of creating our stream. Now I'm running the command again. You'll notice next to the stream status that it's actually active so we're ready to begin working with it. So next we're going to list all the streams that are actually running at this current time. As you see there's our Foo stream so we're good to go. Next we're going to actually send data to our Amazon Kinesis stream. We're gonna send some data called test data. Next we're gonna retrieve that data record. So the first step we need to do is we're gonna actually run the get ShardIterator. Which will provide our ShardInterator which we're gonna pass the next step to actually retrieve the data so it's a two-step process. So this is the get records command and in combination with pasting in the ShardInterator on the last step, this will enable us to retrieve the record. So there's our actual data. You'll notice that you can't understand what it actually says or it doesn't match up with what we originally put in there with the test data. That's because it's encrypted with Base64. So what we're going to need to do is get a decoder, which you can find online, and we can just copy and paste the results into the decoder. There you go. It says "test data" so it matches up perfectly. Creating an Amazon Kinesis stream, putting data to it, retrieving it. And now we're actually gonna delete our stream. So let's check the status of our stream. As you see it has a status of deleting. Let's run it again. Great, it's not found. So that means we've successfully deleted it. Great, so that's the rest of our demo on creating an Amazon Kinesis stream.

About the Author
Learning Paths

Richard Augenti is a DevOps Engineer with 23 years of IT professional experience and 7 years of cloud experience with AWS and Azure. He has been engaged with varying sized projects with clients all across the globe including most sectors. He enjoys finding the best and most efficient way to make things work so, working with automation, cloud technologies, and DevOps has been the perfect fit. When Richard is not engaged with work, he can also be found presenting workshops and talks at user conferences on cloud technologies and other techie talks.