Introduction to Amazon CloudWatch
Amazon CloudWatch Operations
AWS Cost Management Tools
The course is part of this learning path
This section of the SysOps Administrator - Associate learning path introduces you to the different monitoring and reporting services and tools that are relevant to the SOA-C02 exam. We look at both the monitoring of your infrastructure, in addition to the reporting of your bills.
- Understand how Amazon CloudWatch is used to monitoring the performance of your infrastructure
- Learn how to identify anomalies in your infrastructure using Amazon CloudWatch
- Learn how Amazon EventBridge makes it easier to build event-driven applications at scale
- Learn about the different methods of logging that are available
- Understand how to review your costs and optimize them going forward
CloudWatch dashboards are a fantastic way to visualize your data within AWS without having to dig into the nitty-gritty of each individual service.it allows you to quickly display key information at a glance giving you the ability to make decisions about your workload And your processes. These dashboards are created from individual widgets that You can combine together to create graphs and provide detailed information quickly about the topics you desire. even allows you to run queries within these widgets to display even more detailed and specific information.
CloudWatch also has automatic dashboards which are created for you by the service itself. These automatic dashboards work on a service by service basis and pick out some of the key components that you might be interested in.
For example, if you have any ec2 instance already running, there is probably an automatic dashboard that has been created to monitor your ec2 workloads.
I recommend you take a look at some of these automatically created dashboards as they give you a really good understanding of what is available with this service and the types of metrics you can harness to build the perfect dashboard for yourself.
There are two ways that you can create a dashboard. You can either do so visually through the editor or you can create dashboards programmatically and even use them inside cloud formation templates.
Both methods allow you to pick from many different media types called widgets. There are currently 8 flavors of these widgets and they are as follows:
Line charts - A line chart is a type of chart which displays information as a series of data points connected by straight line segments. It is a basic type of chart common in many fields.
Stacked area chart -This type of chart compares the totals of many different subjects within the same graph
Number Widget - Allow you to instantly see the value for a certain metric that you're particularly interested in - this could be as simple as displaying the current number of online instances.
Bar Charts - compares values of multiple types of data within the same graph.
Pie charts - Proportional data in direct relationship to other information fitted within a circle.
Text widget - which is free text with markdown formatting allowing you to add useful information to your dashboards as you see fit
Log tables - which explore results from log insights. Logs Insights enables you to interactively search and analyze your log data in Amazon CloudWatch
And finally, we have alarm statuses: in case you have an alarm set up that you'd like to know immediately if something is going wrong right on this dashboard
One extremely cool feature of CloudWatch dashboards is they allow you to perform math on the metrics you want to display. So if you wanted to see how a graphed metric looked when applying normalization techniques or filters to your data you have the power to do so.
Additionally when working with dashboards are also allowed to aggregate data across multiple sources, like an auto scaling group for example, so if you were interested in seeing how the CPU load was handling overtime across your entire fleet you could create a dashboard that would display that.
Well by now you might be curious how to actually create your own dashboards. It is actually fairly painless to create dashboards with the visual dashboard creation tools provided by AWS within the CloudWatch console.
Creating dashboards in the editor is as simple as drag and dropping and adding new widgets onto a blank canvas. the editor allows you to pick any of the previously mentioned different types of media widgets and place them where you please. Pieces are rearrangeable and can be placed with as much finite controls as you desire. all widgets have a stretchable window view that you can position into specific sizes.
Dashboards can also be written as code giving you programmatic access to all the same information and tools. This means you can also put these code snippets inside cloud formation templates for easy dashboard creation on new accounts or projects. Creating these codified dashboards however is not as easy as it may sound at first. There is a lot of work that goes into testing and making sure your creation functions well.
Your dashboard code is written as a string in JSON formatting and can include anywhere between 0 to 100 separate widget objects. You have to specifically note down the x,y location of your widgets as well as the width and height of each element. That can be a little tedious to set up for the first time, but if you already have a functional blueprint, you can modify that fairly easily.
Here is an example of what your dashboards will look like when written out in code. This structure contains one metric widget ( this is the number widget displaying a metric ) and one text widget. The metric widget follows the CPU utilization of one instance with the id: i-012345.
And the text widget is just a simple title showing off the capabilities of the widget.
When you're building your charts and after you have them completed you have the ability to add annotations to your graphs. This is Helpful for displaying when a certain event has taken place in the past which could help give other members of your team insight and exposure to certain peaks and valleys in your information. Just like writing good code requires comments it's especially important to make sure your graphs and charts also have that advantage.
You can have both horizontal and vertical annotations in your graphs - each having their own purpose. For example, horizontal annotations can denote reasonable top and bottom bounds for a service’s CPU load while vertical annotations are great for noting when a specific event happened in the past.
You also have the ability to link to other dashboards within your own systems or even across accounts. These dashboards don't have to be in the same region either. This is a very powerful tool that helps to centralize operations teams, DevOps, and other service owners who all need to have visibility into the status of your applications.
In order to allow cross-account and cross-region access, you need to enable it within the CloudWatch settings for your account as well as each of the accounts you wish to connect to. You can then link your accounts together, to share CloudWatch data between. These settings can also be activated within the AWS SDK and CLI.
Now, this sharing capability is not all or nothing, you have a few options:
Share a single dashboard and designate specific email addresses and passwords of the people who can view the dashboard.
Share a single dashboard publicly, so that anyone who has the link can view the dashboard.
Share all the CloudWatch dashboards in your account and specify a third-party single sign-on (SSO) provider for dashboard access. All users who are members of this SSO provider's list can access the dashboards in the account. To enable this, you integrate the SSO provider with Amazon Cognito.
CloudWatch Dashboards allow you to have up to three dashboards - each containing up to 50 metrics at no charge. This is more than enough for anyone just practicing or having a few applications they want to monitor. For any more than that however, you will be charged $3 per month per new dashboard you wish to create.
For an enterprise company, that is not too much to spend. However If you are a solo developer or a small shop just starting off - those little 3 dollar charges can add up if you create dashboards willy nilly. So make sure you use your resources appropriately when building dashboards for your services.
Some Dashboard best practices:
Use larger graphs for the most important display metrics - this may seem like a fairly obvious thing to do, but it's important to keep in mind that humans are visual creatures. If you want them to pay attention to something, make it big and obvious.
Layout your dashboards and graph for the average minimum display resolution of your users. - this can help make sure that all relevant data is on screen at one time. This prevents users from missing key information that might be off-screen, which in the case of time-sensitive issues or events could be catastrophic. Most screens these days are able to handle 1920 by 1080 fairly well, however if you know your support staff all look at things on their phones, maybe you can design your dashboards around that instead.
Display time zones within your graphs for time-based data and if multiple operators are expected to be using the same dashboard simultaneously keep the time zone in UTC. This allows people to know at a glance, when an event took place. Its also important during an emergency that all users are working on the same premise in regards to the time the event happened, having to calculate differences in time zones can be frustrating when your customers' satisfaction and your business is on the line.
Default your time interval and datapoint period to whatever is the most common use case.
Avoid plotting too many datapoints within your graphs. Having too much data can slow the dashboard loading time and might reduce the visibility of anomalies.
Annotate your graphs with the relevant alarm thresholds for your services. - this allows your users to understand at a glance if one of your services is about to go over its SLA times or when something terrible is about to happen. Having alarms is great, but never triggering them because you knew something was wrong ahead of time is way better.
Don’t assume your users will know what each metric means - be aggressive with tagging and having descriptions right in the dashboard using text widgets.
Stuart has been working within the IT industry for two decades covering a huge range of topic areas and technologies, from data center and network infrastructure design, to cloud architecture and implementation.
To date, Stuart has created 90+ courses relating to Cloud reaching over 140,000 students, mostly within the AWS category and with a heavy focus on security and compliance.
Stuart is a member of the AWS Community Builders Program for his contributions towards AWS.
He is AWS certified and accredited in addition to being a published author covering topics across the AWS landscape.
In January 2016 Stuart was awarded ‘Expert of the Year Award 2015’ from Experts Exchange for his knowledge share within cloud services to the community.
Stuart enjoys writing about cloud technologies and you will find many of his articles within our blog pages.