This course will provide you with a good foundation to better understand Amazon Kinesis, along with helping you to get started with building streamed solutions. In this course, we'll put a heavier emphasis on hands-on demos along with breaking down the concepts and introducing you to the components that make up Amazon Kinesis.
- People working with Big Data
- Business intelligence
- Demonstrate knowledge of Amazon Kinesis, what are the core aspects of the service (streams, firehose, analytics), their common use cases, big data use cases, and how do you access / integrate with the Kinesis service Demonstrate how to work with and create Kinesis streams
- Demonstrate how to work with Kinesis Firehose and how to use Firehose with Redshift
- Set up monitoring and analysis of the stream services
- Understand how to extract analysis from Kinesis
This Course Includes
- 45 minutes of high-definition video
- Live demonstrations on key course concepts
What You'll Learn
- What is Streaming Data: An overview of streaming data and it’s common uses.
- Setting Up Kinesis Agent: In this demo, we're working with installing the Amazon Kinesis Stream agent.
- Kinesis Streams: An overview of Kinesis Streams, what they do, and common use cases.
- Performing Basic Stream Operations: In this demo, we'll be pulling a basic Amazon Kinesis stream from the command line.
- Firehose: In this lesson, we'll be discussing the fully managed solution, Amazon Kinesis Firehose.
- Firehose Delivery Stream: In this demo, we're going to set up an Amazon Kinesis Firehose stream.
- Testing Delivery Stream: In this lesson, we're going to do a quick follow up to the Firehose stream, and test the data delivery.
- Kinesis Analytics: In this lesson, we'll go over the analytics components of Kinesis.
- Kinesis Analytics Demo: In this demo, we're going to begin working with Amazon Kinesis Analytics.
- Kinesis Features Comparison: In this lesson, we'll compare some products within the Amazon Kinesis suite, as well as some other Amazon services.
- Course Conclusion: A wrap-up and review of the course.
28/05/2019 - Re-record of lectures to improve audio
Welcome to Working with Amazon Kinesis. I'm Richard Augenti and I'll be your instructor for this lesson. In this lesson we'll be discussing Amazon Kinesis Analytics. Amazon Kinesis Analytics is a fully managed analytics solution which works with standard ANSI SQL. Kinesis Analytics enables users to build custom analytics applications to suit their business needs. Out of the box it comes with free build templates that perform a number of functions.
These templates can be further customized as needed. Kinesis Analytics takes advantage of being entirely integrated into the Amazon Kinesis family with Kinesis streams and Firehose, so they fit like a glove. Much like Kinesis Firehouse, Kinesis Analytics is a fully managed service which requires no additional administration, so this frees you up to build out new analytics solutions without the additional IT overhead or maintaining the infrastructure and the personnel to manage it. The scaled building environment is completely automated so there's no concerns with having to worry about not having enough resources to manage your workload. Amazon takes care of that for you.
Additionally, Kinesis Analytics uses standard SQL queries which means you don't have to learn a new programming language to perform queries they've been doing for years in SQL. When it comes to the architecture of Kinesis Analytics there are three main components that comprise the service. The three main components include the input stream which captures the data, the processing point which runs the SQL queries against the capture and input stream, and its delivery of the processed data which can be forwarded to other analytic solutions to perform actions such as alerts in real time.
The processed results can be forwarded to other Amazon services such as S3, Redshift, the elastic search, or other custom destinations. When you create an application input you connect the streaming input to an in-application stream which is created. The source data is continuously flowing into in-application streams which acts like a table that can be queried against those SQL statements. There's a special column that is created called the ROWTIME which reflects the timestamp for when the data was inserted into the application table. SQL queries are continuously running against your in-application table.
Many times you are seeking to get results within a timeframe, thus the need to utilize ROWTIME or other time-based columns. This type of query is referred to as windowed SQL or windowed queries. You can have multiple in-application streams which can combine to form a stream join. As you can see, Kinesis Analytics provides a deep set of querying capabilities to assist with getting our desired results. Kinesis Analytics leverages logging and monitoring solutions within the Amazon family.
These solutions can include CloudWatch alarms, CloudWatch logs, CloudWatch events, CloudTrail monitoring and trusted advisor. The ability to harness services like this across the Amazon environment is a true testament of Amazon being a one-stop shop for all your business technology needs. Well, that wraps up our lesson on Amazon Kinesis Analytics. I'm Richard Augenti and I'll see you in the next lesson.
About the Author
Richard Augenti is a DevOps Engineer with 23 years of IT professional experience and 7 years of cloud experience with AWS and Azure. He has been engaged with varying sized projects with clients all across the globe including most sectors. He enjoys finding the best and most efficient way to make things work so, working with automation, cloud technologies, and DevOps has been the perfect fit. When Richard is not engaged with work, he can also be found presenting workshops and talks at user conferences on cloud technologies and other techie talks.