In course one of the AWS Big Data Specialty Data Collection learning path we explain the various data collection methods and techniques for determining the operational characteristics of a collection system. We explore how to define a collection system able to handle the frequency of data change and the type of data being ingested. We identify how to enforce data properties such as order, data structure, and metadata, and to ensure the durability and availability for our collection approach.
Learning Objectives
- Recognize and explain the operational characteristics of a collection system.
- Recognize and explain how a collection system can be designed to handle the frequency of data change and the type of data being ingested.
- Recognize and identify properties that may need to be enforced by a collection system.
Intended Audience
This course is intended for students looking to increase their knowledge of data collection methods and techniques with big data solutions.
Prerequisites
While there are no formal prerequisites, students will benefit from having a basic understanding of analytics services available in AWS. Please take a look at our Analytics Fundamentals for AWS
This Course Includes
- 45 minutes of high-definition videos
- Live hands-on demos
What You'll Learn
- Introduction to Collecting Data: In this lesson, we'll prepare you for what we'll be covering in the course; the Big Data collection services of AWS Data Pipeline, Amazon Kinesis, and AWS Snowball.
- Introduction to Data Pipeline: In this lesson, we'll discuss the basics of Data Pipeline.
- AWS Data Pipeline Architecture: In this lesson, we'll go into more detail about the architecture that underpins the AWS Data Pipeline Big Data Service.
- AWS Data Pipeline Core Concepts: In this lesson, we'll discuss how we define data nodes, access, activities, schedules, and resources.
- AWS Data Pipeline Reference Architecture: In this lesson, we'll look at a real-life scenario of how data pipeline can be used.
- Introduction to AWS Kinesis: In this lesson, we'll take a top-level view of Kinesis and its uses.
- Kinesis Streams Architecture: In this lesson, we'll look at the architecture that underpins Kinesis.
- Kinesis Streams Core Concepts: In this lesson, we'll dig deeper into the data records.
- Kinesis Streams Firehose Architecture: In this lesson, we'll look at firehose architecture and the differences between it and Amazon Kinesis Streams.
- Firehose Core Concepts: Let's take a deeper look at some details about the Firehose service.
- Kinesis Wrap-Up: In this summary, we'll look at the differences between Kinesis and Firehose.
- Introduction to Snowball: Overview of the Snowball Service.
- Snowball Architecture: Let's have a look at the architecture that underpins the AWS Snowball big data service
- Snowball Core Concepts: In this lesson, we'll look at the details of how Snowball is engineered to support data transfer.
- Snowball Wrap-Up: A brief summary of Snowball and our course.
Hello, and welcome to another Big Data on AWS course from Cloud Academy. In this course, we focus on the Amazon Big Data services which are designed to collect data. This course is part of a larger learning path that covers the broad range of Big Data services available from AWS.
This course assumes you have a good understanding of cloud computing in AWS, and that you are proficient with provisioning and using services within AWS. Ideally, you will also have some background understanding of Big Data. There are a large number of AWS Big Data services available and this course is designed to provide the initial core concepts required for each of these services, and to assist people in passing the AWS Big Data Specialist exam.
A little bit about me. My name is Shane Gibson, and I have worked in the area of data and business intelligence for over 20 years. And for the last three years, I've been focusing on how we can use agile processes and cloud computing technologies to accelerate the delivery of data and content to our users. I was born and still live in New Zealand. I love craft beer and good coffee. You can learn more about me by following either on my Twitter or my LinkedIn. At the end of this course, you'll be able to describe in detail how Amazon Big Data services can be used to collect data within a Big Data solution. In this Big Data on AWS learning path, we cover many AWS Big Data services that can be used to collect, store, process, analyze, visualize and secure Big Data.
In this course, we provide three modules which cover the Big Data collection services of AWS Data Pipeline, Amazon Kinesis and AWS Snowball. Each of these three Big Data collection services can be used on their own, or in combination with each other to provide processing capabilities for your Big Data solution. Each of these collection services have specific strengths that make them more suitable for the collection of different types and volumes of data, and we discuss these as we progress through the course. In each of the modules we cover, which processing and storage patterns that collections service fits within, the architecture of the service, as well as the core concepts that help you understand that service in detail.
We also cover the service limits for each service where applicable. At the end of the three modules, we will have a wrap up with a quick overview of the reference architecture which uses these services. So let's begin and find out how we can collect Big Data using Amazon Web Services capabilities.
Shane has been emerged in the world of data, analytics and business intelligence for over 20 years, and for the last few years he has been focusing on how Agile processes and cloud computing technologies can be used to accelerate the delivery of data and content to users.
He is an avid user of the AWS cloud platform to help deliver this capability with increased speed and decreased costs. In fact its often hard to shut him up when he is talking about the innovative solutions that AWS can help you to create, or how cool the latest AWS feature is.
Shane hails from the far end of the earth, Wellington New Zealand, a place famous for Hobbits and Kiwifruit. However your more likely to see him partake of a good long black or an even better craft beer.