Metrics and Anomaly Detection
The course is part of this learning path
This course covers Amazon CloudWatch and CloudWatch Alarms using Anomaly Detection.
Amazon CloudWatch is the monitoring and observability service from AWS. The phrase Anomaly Detection implies that this feature is used to detect outliers but this is an understatement. It is a feature of CloudWatch that uses Machine Learning to automate the creation of alarms and their related thresholds.
This course includes a review of Amazon CloudWatch and the challenges of setting and maintaining alarms. It covers how machine learning with Anomaly Detection helps setting alarms and managing/maintaining their thresholds.
You'll learn how to create a CloudWatch Alarm using Anomaly Detection and learn what types of metrics are suitable for use with Anomaly Detection.
- Gain a high level of Amazon CloudWatch
- Review how monitored metrics go into an ALARM state
- Learn about the challenges of creating CloudWatch Alarms and the benefits of using machine learning in alarm management
- Know how to create a CloudWatch Alarm using Anomaly Detection
- Learn what types of metrics are suitable for use with Anomaly Detection
This course is for anyone who wants or needs to create CloudWatch Alarms that are almost completely automated.
To get the most out of this course, you should have some experience running workloads in the AWS cloud, know what Amazon CloudWatch is, know how to create CloudWatch Alarms, and how to trigger an action or notification based on an Alarm's state.
Amazon CloudWatch is a monitoring and observability service from AWS. It can provide a unified view of the performance and health of AWS resources as well as applications & services running both inside the AWS cloud and on-premises systems.
Creating alarms using Amazon CloudWatch is both an art and a science. It's a science because it uses numbers to measure activity. But, because you need to get a feeling for what that activity is, over time, it's also an art.
Creating alarms using Anomaly Detection takes the guesswork out of alarm creation by using machine learning algorithms to identify data points, events, or observations that do not conform to a typical pattern or expected behavior.
It continuously analyzes your system, learns the normal baseline of applications, and reveals anomalies in their behavior. It works best when metrics have a discernible pattern or trend.
The words, "discernible pattern or trend" are important. The Anomaly Detection model cannot predict one-time events like Black Friday or holiday shopping seasons.
If you're not from the United States, Black Friday is the day after Thanksgiving and is the traditional start to the Christmas shopping season. There are some people that love that kind of stress in their lives. I don't know any of them but I know they exist because they're the ones out shopping.
Anyway, back to CloudWatch.
If data points are missing, such as the case when a new EC2 instance is created or an existing instance has been off for days or weeks, it will take some time before the model is accurate.
Similarly, if an instance has been idle--and wasting money--the model will be built using inaccurate data. For conditions like this--as well as for stress testing and other behavior that is out of the ordinary--you should exclude time ranges.
With CloudWatch Anomaly Detection, it's possible to create an alarm that can adapt to metric trends as well as adapt to the dynamic nature of system and application behavior such as time-of-day utilization peaks without requiring user intervention.
As a reminder, CloudWatch Anomaly Detection has a number of key features.
It can learn and model the expected behavior of a metric based on prior data.
It will calculate expected predicted confidence values and generate an Anomaly Detection confidence band. This band is based on the normal ranges generated by the model and metric values that fall outside the band are considered anomalies.
Alarms can be created based on this normal pattern and be triggered when they are "Outside the band," "Greater than the band," or "Lower than the band."
Alarms are based on a prediction value which continues to adapt over time.
Amazon CloudWatch Anomaly Detection can be configured using the AWS Console and it also has AWS API support. This means it can be configured using the AWS CLI, the AWS SDKs, and AWS CloudFormation.
I think it's a big step in the process of automating alarms. Some human intervention is still required. However, it makes the process of creating alarms much more objective.
This brings me to the end of this course. But, it really isn’t the end. There’s more to learn and experience with monitoring and observing cloud computing. I hope you found this material useful and that you can use it to take your next steps on your cloud journey.
Whether I want to or not, it seems that I learn something new every day.
Part of the reason why is due to the rate of change in the cloud and part of it is having the freedom to explore and experiment without the constraints of a physical data center.
Your comments, questions, and suggestions are of great value to us at Cloud Academy. We use this information to improve our courses in terms of both content and delivery.
Please reach out to us by emailing firstname.lastname@example.org. I'd love to hear your thoughts about this course as well as what you'd like to learn about in the future.
I'm Stephen Cole with Cloud Academy and thank you for watching!
Stephen is the AWS Certification Specialist at Cloud Academy. His content focuses heavily on topics related to certification on Amazon Web Services technologies. He loves teaching and believes that there are no shortcuts to certification but it is possible to find the right path and course of study.
Stephen has worked in IT for over 25 years in roles ranging from tech support to systems engineering. At one point, he taught computer network technology at a community college in Washington state.
Before coming to Cloud Academy, Stephen worked as a trainer and curriculum developer at AWS and brings a wealth of knowledge and experience in cloud technologies.
In his spare time, Stephen enjoys reading, sudoku, gaming, and modern square dancing.