Introduction to Amazon CloudWatch
Amazon CloudWatch Operations
AWS Cost Management Tools
The course is part of this learning path
This section of the SysOps Administrator - Associate learning path introduces you to the different monitoring and reporting services and tools that are relevant to the SOA-C02 exam. We look at both the monitoring of your infrastructure, in addition to the reporting of your bills.
- Understand how Amazon CloudWatch is used to monitoring the performance of your infrastructure
- Learn how to identify anomalies in your infrastructure using Amazon CloudWatch
- Learn how Amazon EventBridge makes it easier to build event-driven applications at scale
- Learn about the different methods of logging that are available
- Understand how to review your costs and optimize them going forward
Hello, and welcome to this lecture. Where we will be taking a look at the components that make up the AWS Config Service and the functions that each of them carry in delivering the service. To understand how to get the most of the service, it's important to understand how the services piece together and how it works. So let's start by identifying the key components. Following this, we will then break each of them down to understand their role and how they sit within the service.
The following identifies the main components to the service. AWS resources, configuration items, configuration streams, configuration history, configuration snapshots, configuration recorder, config rules, resource relationships, SNS topics, S3 buckets and AWS Config permissions. Let's now take a look at each of these and turn in more detail. Starting with AWS resources.
AWS resources are typically classed as objects that can be created, updated or deleted from within the AWS Management Console or programmatically, using the AWS CLI or supported STK. AWS Config records changes to supported AWS resources within a specific Region. For the current list of supported services and resources, please follow the link on screen.
A configuration item or CI as is known, is a key component of AWS Config. It is comprised of a JSON file that holds the configuration information, relationship information, and other metadata as a point in time snapshot view of a supported resource. All the information that AWS Config can record for resource, is captured within the CI. A CI is created every time a supported resource has a change made to its configuration in any way.
In addition to recording the details of the affected resource, AWS Config will also record CIs for any directly related resources to ensure the change did not affect those resources too. For example, if there was a rule change to a security group, additional rules would be added with new parts. AWS Config would record all CI information for that resource, but it would also give a CI information for any instances that were a part of that security group. The CIs would then be sent to a configuration stream, which we will cover next.
Because it captures so much data, these CIs are used by other features and components of AWS Config, such as configuration history. CIs are used to look up all the changes that have been made to a resource.
Configuration Streams. CIs are sent to an SNS topic to enable analysis of the data. Configuration snapshots. CIs are used to create a point in time snapshot of all supported resources. Following CIs, let's move on to configuration streams. When a create, update or delete API call is made against a supported AWS Config resource, as we now know a new CI is created along with additional CIs for any resources that are related to the original modified resource.
These CIs are then sent to a configuration stream, and this stream is in the form of an SNS topic. However, this stream is also used by AWS Config to send information when other events occur such as when the configuration history for a resource was delivered to your account, when a configuration snapshot was started and delivered to your account, when the state of your resource compliance changes against any config rules that have been configured, when evaluation begins for rules against resources, and when AWS Config found to deliver notifications to your account.
The configuration history uses configuration items to collate and produce a history of changes to a particular resource. This allows you to see the complete set of changes made to resource over a set period of time. The information can be accessed either programmatically through the AWS CLI using the following command. You can also specify the resource type. So for example, if you wanted to look at the configuration history for a Subnet, you could enter the following into the AWS CLI. Or you could access the history via the AWS Management Console.
Additionally, AWS Config also sends a configuration history file for each resource type to an S3 bucket that is selected during the setup of AWS Config. This configuration file is typically delivered every six hours. Unlike our site, it contains all CI changes for all resources of a particular type. For example, there would be one configuration history file covering six hours for all RDS DB instance changes in one Region.
The configuration snapshot also uses configuration items in its production. The configuration snapshot will take a point in time snapshot on all supported resources configured for that Region. It will generate CIs for each resource in your AWS account for a specific Region, and this configuration snapshot can then be sent to an S3 bucket. Alternatively, this information can be viewed by the AWS Management Console.
Now let's take a look at the configuration recorder. So this component of AWS Config can be seen as the engine of the service. This element is responsible for recording all of the changes to the supported resources within your account, and generating the configuration items. By default, the configuration recorder is automatically enabled and started when you first configure AWS Config. However, it is something that you can stop and then will start again at a later point. When you stop it, AWS Config will no longer track and record changes to your supported of resources.
When you first configure AWS Config in the AWS Management Console, you are asked which resource types you would like to record. This is essentially setting up and creating the configuration recorder, as this information is required to enable the configuration recorder to start. If you only select to record specific resource types, then it will capture all creates, changes and deletes for those resource types, and will create a CI record for each occurrence. However, the configuration recorder will still record all creates and deletes of supported resource types within that Region, but no changes to those resources will be recorded.
Also, if you stop and change your configuration recorder settings to, perhaps, remove certain resource types from being recorded, then all captured information for those resources up to that point will remain, and you will be able to view the history with all data from the previous CIs that have been captured.
AWS Config rules are a great way to help you enforce specific compliance checks and controls across your resources and allows you to adopt an ideal deployment specification for each of your resource types. Each rule is essentially a Lambda function, that when called upon, evaluates the resource and carries out some simple logic to determine the compliance result with the rule.
Each time a change is made to one of your supportive resources, AWS Config will check the compliance against any config rules that you have in place. If there was a violation against these rules, then AWS Config will send a message to the configuration stream by SNS, and the resource will be marked as noncompliant.
It's important to note that this does not mean the resource will be taken out of service or it will stop working, it will continue to operate exactly as it is with its new configuration. AWS Config simply alerts you that there's a violation and it's up to you to take the appropriate action. These rules can be custom defined or selected from a predefined list of AWS managed rules that AWS has created on your behalf.
Being able to create your own rules, allows you to adopt best practices that you may have internally within your own enterprise or with other security best practices. By allowing AWS Config to monitor resources at this level, it adds another level of automation, helping to prevent misconfigurations made by human error being left, which could lead to a security risk or even worse, a breach. I highly recommend using config rules for maintaining security checks and configurations.
AWS have a number of predefined rules that fall under the security umbrella that are ready to use. For example, Rds-storage-encrypted, this checks whether storage encryption is activated by your RDS database instances. Encrypted-volumes, this checks to see if any EBS volumes that are in attached state, are encrypted. Root-account-mfa-enabled, this checks whether your root account of your AWS account requires multi-factor authentication for console sign-in. IAM-user-no-policy-check, this checks that none of your IAM users have policies attached.
Best practice dictates that permission should be provided by roles or groups. The great thing about these predefined rules is that you can also edit them to make subtle parameter changes as needed. As you can see, being able to ask either AWS Config to check your resources compliance with rules such as these are invaluable. And when you couple this with being able to create your own custom rules, the scope and potential of automating compliance is huge across your supported resources.
As a part of the CI for a particular resource, for example, in EBS volume, AWS Config identifies relationships with other resources from that resource. In this case, it might be the EC2 instance that the volume is attached to. This allows AWS Config to build a logical mapping of resources and how they connect. This mapping of relationships allows you to quickly jump to other linked resources within the AWS Management Console to view their configuration history and CI data. This is very useful if you're trying to troubleshoot an issue and pinpoint where the source of an incident may be. For a full listing of available relationship types, see the link on screen.
As we have already seen, an SNS topic is used as a configuration stream for notifications of various events triggered by AWS Config. You can have various end points associated to the SNS stream. Best practice in the case that you should use SQS and then programmatically analyze the results via SQS. The S3 bucket that was selected at the time of configuration, is used to store all that configuration history files that are generated for each resource type, which happens every six hours.
Also, any configuration snapshots that are taken are also stored within the same S3 bucket. The configuration details used for both SNS and S3 are classed as the AWS Config delivery channel, by which data can be sent to other services. When setting up AWS Config, you're required to select an IAM role. This role is required to allow AWS Config to obtain encrypt permissions to carry out and perform a number of functions.
For example, AWS Config will need read only access to all the supported resources within your account so it can retrieve data for the configuration items. Also, we now know that AWS Config uses SNS and S3, both the streams and storage of the configuration history files and snapshots. So AWS Config requires the relevant permission to allow it to send data to these services.
That now covers the key elements of AWS Config. And so I hope it makes it clear as to how each part plays a role within the service. To reiterate, let's take a bulleted point look at how it all comes together.
When you first turn on AWS Config, you will configure the elements required for the configuration recorder to begin capturing and recording data. AWS Config will then discover all of your supported resources based upon the details entered within the configuration recorder within that Region. When a create, change or delete against a resource is made, a CI will be created for this change, and a notification will be sent the configuration stream regarding the new CI. Also, following the CI, AWS Config will check its current config rules to evaluate if the changes made the resource noncompliant. If the evaluate state change for the resource, a notification will be sent to the configuration stream. Your config rules can also be configured to run periodically, and so that will run at a set given time period regardless if there have been any changes to resources. If a configuration snapshot is taken, AWS Config will create a point in time snapshot of the resources with new CIs and deliver this configuration to your specified S3 bucket within the configuration recorder. After six hours has passed from turning on AWS Config, a configuration history file will be created for each resource type, and again, sent to your specified S3 bucket. The configuration file will be sent every six hours from that point.
Stuart has been working within the IT industry for two decades covering a huge range of topic areas and technologies, from data center and network infrastructure design, to cloud architecture and implementation.
To date, Stuart has created 90+ courses relating to Cloud reaching over 140,000 students, mostly within the AWS category and with a heavy focus on security and compliance.
Stuart is a member of the AWS Community Builders Program for his contributions towards AWS.
He is AWS certified and accredited in addition to being a published author covering topics across the AWS landscape.
In January 2016 Stuart was awarded ‘Expert of the Year Award 2015’ from Experts Exchange for his knowledge share within cloud services to the community.
Stuart enjoys writing about cloud technologies and you will find many of his articles within our blog pages.