1. Home
  2. Training Library
  3. Serverless, Component Decoupling, and Solution Architectures (SAP-C02)

Provisioning an Amazon MSK Cluster

Contents

keyboard_tab
Course Introduction
1
Introduction
PREVIEW2m 26s
Utilizing Managed Services and Serverless Architectures to Minimize Cost
Decoupled Architecture
Amazon API Gateway
10
Advanced API Gateway
PREVIEW11m 29s
Amazon Elastic Map Reduce
18
Introduction to EMR
PREVIEW1m 46s
Amazon EventBridge
26
EventBridge
PREVIEW7m 58s
Design considerations
39

The course is part of this learning path

Start course
Difficulty
Intermediate
Duration
4h 43m
Students
80
Ratings
3/5
starstarstarstar-borderstar-border
Description

This section of the AWS Certified Solutions Architect - Professional learning path introduces common AWS solution architectures relevant to the AWS Certified Solutions Architect - Professional exam and the services that support them. These services form a core component of running resilient and performant architectures. 

Want more? Try a Lab Playground or do a Lab Challenge!

Learning Objectives

  • Learn how to utilize managed services and serverless architectures to minimize cost
  • Understand how to use AWS services to process streaming data
  • Discover AWS services that support mobile app development
  • Understand when to utilize serverless services within your AWS solutions
  • Learn which AWS services to use when building a decoupled architecture
Transcript

There are two elements you need to provision when creating your Amazon MSK clusters: Broker instances and Broker storage. A Broker instance is a worker node that helps to manage the Kafka cluster. Your clusters can have multiple brokers, but can also operate as a single node, if necessary. Broker instances can be run within the same availability zone or across many availability zones to create a highly available cluster, something that many architectures will require.

One of the large benefits of using Amazon MSK over a roll-your-own version of Kafka is that Amazon will keep an eye on these broker nodes and replace them if they become unhealthy. Otherwise, you would have to manage all of that yourself. Broker storage is where all of your data will be kept that comes streaming into Amazon MSK.

Within Amazon, this storage is housed within EBS volumes, and gains all the protections that EBS provides, like durability and fault tolerance. You can also encrypt these data volumes using Amazon EBS Server-side encryption and AWS KMS, the Key Management Service. An interesting note about storage however, is that once you have assigned your Broker storage, you can only increase the amount of storage, it can not be decreased. The broker's storage can be autoscaled upwards if you do reach your maximum capacity.

If you already have a Kafka cluster that you are managing yourself, either on-premises or within the cloud, you can migrate over to Amazon MSK. There are many third-party and open-source tools like MirrorMaker which can help replicate the data from your Kafka cluster into Amazon MSK.

About the Author
Students
37985
Courses
26
Learning Paths
20

Danny has over 20 years of IT experience as a software developer, cloud engineer, and technical trainer. After attending a conference on cloud computing in 2009, he knew he wanted to build his career around what was still a very new, emerging technology at the time — and share this transformational knowledge with others. He has spoken to IT professional audiences at local, regional, and national user groups and conferences. He has delivered in-person classroom and virtual training, interactive webinars, and authored video training courses covering many different technologies, including Amazon Web Services. He currently has six active AWS certifications, including certifications at the Professional and Specialty level.