Introduction to Amazon CloudWatch
Amazon CloudWatch Operations
AWS Cost Management Tools
The course is part of this learning path
This section of the SysOps Administrator - Associate learning path introduces you to the different monitoring and reporting services and tools that are relevant to the SOA-C02 exam. We look at both the monitoring of your infrastructure, in addition to the reporting of your bills.
- Understand how Amazon CloudWatch is used to monitoring the performance of your infrastructure
- Learn how to identify anomalies in your infrastructure using Amazon CloudWatch
- Learn how Amazon EventBridge makes it easier to build event-driven applications at scale
- Learn about the different methods of logging that are available
- Understand how to review your costs and optimize them going forward
Hello, and welcome to this lecture. Where we are to discuss the different features and components of CloudTrail and how they work together to provide a customizable API call tracking and monitoring solution.
Firstly, let's look under the hood of CloudTrail to see what makes up the core features and components that create the service.
- Trails. These are the building blocks of the service. You can create many different Trails containing different configurations related to API requests that you want to capture.
- S3. S3 is used by default to store the CloudTrail log files and a dedicated S3 bucket is required during the creation of a new Trail.
- Logs. Logs are created by AWS CloudTrail and record all events captured. A new log file is created approximately every five minutes and once processed, it is delivered to an S3 bucket as defined by its Trail configuration. If no API calls to be made, then no logs will be delivered.
- KMS. The use of AWS KMS is an optional element of CloudTrail, but it allows additional encryption to be added to your log files when stored on S3.
- SNS. SNS is also an optional component for CloudTrail, but it allows for you to create notifications. For example, when a new log file is delivered to S3 SNS could notify someone or a team via an email, or it could be used in conjunction with CloudWatch. When metric thresholds have been reached.
- CloudWatch logs. Again, this is another optional component. For AWS CloudTrail allows you to deliver its logs to AWS as CloudWatch logs, as well as S3 for specific monitoring metrics to take place.
- Events Selectors. Events Selectors allowed to add a level of customization to the type of API requests, you want the corresponding trails to capture.
- Tags, Tags allow you to assign your own metadata to your Trail. For example, you could add a project or department tag indicating which project or department the trail relates to.
- Events. For every API request that is captured by CloudTrail it is recorded as an event in a CloudTrail log file. API Activity Filters. These are search filters that can be applied against your API activity history in the management console for create, modify and delete API calls. These events are held in the management console for seven days. Even if the Trails itself is stopped or deleted.
Okay, so I've now covered the different components that essentially build CloudTrail. Let me now introduce you to the process at a high level of how all of this fits together and in what order. The very first step is to create a Trail. If no Trail exists, then CloudTrail does not know what API cost to capture from which region and which services, global or otherwise. During this Trail creation, you will need to specify an S3 bucket for your log files to be delivered to you. This can be an existing bucket or new bucket, which can be created at this stage.
You will then have an option to encrypt your log files with the key management service KMS, if required. Also if required, you can configure the Simple Notification Service SNS to notify you when new log files are delivered. If you want to add another layer of security, integrity, then you can enable log file validation, which will ensure your logs have not been modified or tampered with since they delivered to S3. Your Trail is now ready to be turned on and created.
Once it has been created, you are able to make further configurational changes that are not available during the Trail creation itself. Upon selection of your new Trail you will have the option to configure CloudWatch logs, which allows you to deliver your cloud Trails log files to CloudWatch in addition to S3. This allows you to create CloudWatch monitoring metrics against specific API calls and will receive notification from SNS when custom thresholds are reached.
Another optional configurable element is that of CloudTrail Event Selectors. This allows you to specify the types of events, management or data that CloudTrail logs. Finally, at the last element of your newly created Trail configuration, there is the ability to add tags just as you would with other resources within AWS. At this point, your Trail is configured and actively recording API calls as per your configuration.
For every API call that matches the requirement of your Trail, it will be captured and recorded in a log file as an event. Each API call will be recorded as a new event. Once you've captured the data, you may need to find a particular event quickly, maybe for security reasons. This can be achieved using API Activity Filters, which can be found within the CloudTrail service from the management console.
So from a high-level perspective, we know how a Trail is configured, but what happens when an API matching a Trail is called upon by user or service? Let's take a quick look.
A user or service calls upon an API. Next CloudTrail checks to see if this API call matches any configured Trails. If a match is found CloudTrail records the API as an event within its current log file. It also associates other identical metadata mentioned earlier. Eventually, the event with the log file will be delivered to S3 and possibly CloudWatch logs, depending on the Trail configuration. If it is sent to CloudWatch logs, the log file will be monitored by any configured metrics. When in S3, the log file will be stored and the default server-side encryption SSE, and unless KMS has been configured for increased security measures with the associated Trail. If any S3 lifecycle cycle rules are applied to the bucket, then over time, the log file may be archived to a different storage class or even glassier.
That essentially concludes the elements of CloudTrail and how they work together.
Stuart has been working within the IT industry for two decades covering a huge range of topic areas and technologies, from data center and network infrastructure design, to cloud architecture and implementation.
To date, Stuart has created 90+ courses relating to Cloud reaching over 140,000 students, mostly within the AWS category and with a heavy focus on security and compliance.
Stuart is a member of the AWS Community Builders Program for his contributions towards AWS.
He is AWS certified and accredited in addition to being a published author covering topics across the AWS landscape.
In January 2016 Stuart was awarded ‘Expert of the Year Award 2015’ from Experts Exchange for his knowledge share within cloud services to the community.
Stuart enjoys writing about cloud technologies and you will find many of his articles within our blog pages.