Transcript

Hello and welcome to this lecture regarding AWS Config. AWS Config is a great security and compliance tool that integrates well with many other AWS services. As you may or may not know, AWS Config can perform the following functions. It can capture resource changes, so any change to a resource supported by Config can be recorded, which will record what changed along with other useful metadata or held within a file known as the Configuration Item, the CI. It can act as a resource inventory. AWS Config can discover supportive resources running within your environment, allowing you to see data about that resource type. It can store configuration history for individual resources. The service will record and hold all existing changes that have happened against the resource, providing a useful history record of changes. It can provide a snapshot in time of current resource configurations where an entire snapshot of all supported resources within a region can be captured that will detail their current configurations with all related metadata. It can enable notifications of when a change has occurred on a resource. So SNS is used with AWS Config to capture a configuration's stream of changes enabling you to process and analyze the changes to resources. It can provide information on who made the change and when through AWS CloudTrail Integration. AWS CloudTrail is used AWS Config to help you identify who made the change and when and with which API. It can enforce rules that checks the compliancy of your resource against specific controls. Predefined and custom rules can be configured within AWS Config, allowing you to check resources compliance against these rules. It can perform security analysis within your AWS environment. A number of security resources can be recorded and when this is coupled with rules relating to security such as encryption checks, this can become a powerful analysis tool. And finally it can provide relationship connectivity information between resources. The AWS Management Console provides a great relationship allowing you to quickly see and identify which resources are related to any other resource. For example, when looking at an EBS volume, you'll be able to see which EC2 instance it is connected to. 

From a logging perspective, the ability to store configuration history is highly valuable. So let me look into this a little further. The configuration history uses Configuration Items, CIs, to collate and produce a history of changes to a particular resource. This allows you to see the complete set of changes made to a resource over a set period of time. The information can be accessed either programmatically though the AWS CLI using the following command, or you can also specify the resource type. So for example, if you wanted to look at the configuration history for a subnet you could enter the following into the CLI. Or you could also access this history via the AWS Management Console. Additionally, AWS Config also sends a configuration history file for each resource type to an S3 bucket that is selected during the setup of AWS Config. This configuration file is typically delivered every six hours and it contains all CI changes for all resources of a particular type. For example, there'll be one configuration history file covering six hours for all RDS DB instant changes in one region. A Configuration Item, or CI as it's known, is a key component of AWS Config. It is comprised of a JSON file that holds the configuration information, relationship information, and other metadata as point in time snapshot view of a supported resource. All the information that AWS Config can record for resource is captured within the CI.

 A CI is created every time a supported resource has a change made to its configuration in any way. In addition to recording the details of the affected resource, AWS Config will also record CIs for any directly related resources to ensure the change did not affect those resources, too. For example, if there was a rule change to a security group, perhaps additional rules were added with new ports. AWS Config will record all CI information for that resource. But it also gathers CI information for any instances that were part of that security group. These will then be sent to the configuration stream. As so much data is gathered within these CIs, it's important we look at these in further detail. 

So for every CI generated there will be five different sections. Firstly, metadata. This essentially contains details about the configuration item itself. So within this metadata we have both a version ID and a configuration ID, which uniquely identifies the CI. In addition to this, other information includes an MD5Hash that allows you to compare other CIs already recorded against the same resource as well as ensuring there are no duplications. And then we have the time of the capture and a state ID, which puts the CIs for a particular resource into an order of sequence. 

Attributes, this holds common attribute information against the actual resource. Within this section we also have a resource ID and any key-value tags that are associated to the resource. The resource type is also listed. For example, if this was a CI for an EC2 instance, the resource types listed could be the network interface or the EIP for that EC2 instance. The Amazon Resource Name, the ARN, for the resource would also see shown along with the availability zone that the resource belonged to. Bear in mind that for services and resources that are not fixed to a particular availability zone, such as IAM, then this section would not be applicable. Lastly, the time that the resource was created would also be given. 

Relationships, this holds information for any connected relationships that the resource may have. So within this section it would show a clear description of any relationship to other resources that this resource had. For example, if the CI was from EC2 instance the relationship instance may show the connection to a VPC along with a subnet that the EC2 instance resides in. 

Current configuration. This will display the same information that would be generated if you were to perform or describe or list API call made by the AWS CLI. AWS Config uses the same API call to get the same information. So depending on the API call and resource, different information will be returned, which is resource specific. 

Related events, and this relates to AWS CloudTrail. This will display the AWS CloudTrail event ID that is related to the change that triggered the creation of this CI. There is a new CI made for every change made against a resource. As a result, a different CloudTrail event ID will be created. This allows you to deep dive into who or what and when made the change that triggered the CI. A great feature allowing for some great analysis to be taken, specifically when this affects security resources. 

As you can see, the configuration item is a fundamental aspect of AWS Config when recording changes and data made to a supported resource. The configuration history file is stored on the S3 bucket that was selected at the time of configuration and is used to store all the configuration history files that are generated for each resource type, which happens every six hours. If you have multiple AWS accounts you may want to aggregate your configuration history files into the same S3 bucket for your primary account. However, you'll need to grant write access for this service principle, config.amazonaws.com, and your secondary accounts with write access to the S3 bucket in your primary account. Let me now demonstrate how to use the AWS Management Console to view configuration details and history within my account. 

Start of demonstration

Okay, so I've just logged into my AWS account, and I'm going to go up to services and you can see the AWS Config is under management tools. So if we select config, and this is the first splash screen you'll be presented with if you don't have AWS Config started as yet. So, let's get started by clicking on the get started button. And then there's three steps to setting AWS Config up. 

So the first step is configuring the settings. And this is where we essentially start the configuration recorder, by identifying what resources we'd like included in our AWS Config configuration. So starting from the top, we can either select all resources within this region, remember, AWS Config is region specific. So we can choose to record all the resources supported within this region, and currently I'm in the London region. And we can choose to include global services such as IAM, etc. But for this demonstration we'll just choose to record all resources in this region. 

Here you can select specific types if you're required. So if I unticked this we can then go down to this dropdown box and select specific resource types that we'd like set up for this configuration. So you can be quite specific. Selecting the load balancer or different elements of IAM and again different elements of RDS, etc. So there's a list that it can go through if you just want to record a specific resource type. But like I said for this demonstration I'm just going to select all resources. 

Next we need to define our S3 bucket. And this is where our configuration history and snapshot files are stored. So we can ask AWS Config to just create a new bucket for us and it will prefix it with this bucket name here or we can choose an existing bucket from my account, by selecting the dropdown list or we can select a bucket from another account. So if you had multiple accounts you can send the configuration history files and snapshot files for all those accounts to a single primary AWS account in a single bucket. So for this example, though, for this demonstration I'm just going to ask AWS Config to create a new bucket for us. Config bucket Cloud Academy. 

Now moving down to the SNS topic. Now this is the configuration stream. So for any events that AWS Config picks up it will send it to this SNS topic for the configuration stream. Now we can create a new topic, and again, AWS Config will just give us topic name here or we can select to choose another topic from your account or again from another account. For this demonstration we'll just ask AWS Config to create the topic name for us, and I'll just add in Cloud Academy. 

Now finally we have the AWS Config role, and this is where we spoke about the different permissions. And this is required to allow AWS Config to send the configuration history file and snapshots to S3 as well as having access to send events to the SNS topic. I'm not forgetting enabling AWS Config to be able to kind of go out and poll your resources as a describe or list API call to get the details of those resources. And again, for this demonstration, I'm just going to ask AWS Config to create a role for us. I'll just add Cloud Academy on the end. So that's settings. So this screen is essentially your configuration recorder. By setting this up you're asking AWS Config to start recording. We have set up the S3 bucket to allow our configuration history files and our snapshots files to be captured. And we have created our configuration stream by configuring an SNS topic. And we have also allowed AWS Config to have the necessary permissions to carry out those functions. So once that's all set up we'll click on next. 

And here we have a set of predefined AWS managed config roles. This is a number of templates that you can select to check the compliance of your resources against these rules. So for now I'm just going to click on skip. And here we're on the final stage which is review. So we can see here the resource types we've selected, which are all resources, and we've not chosen to include global resources. We have our S3 bucket set up for our history files and snapshots and we have our SNS topic for our configuration stream configured, and we have the permissions given by the AWS Config role. And that's essentially the elements to setting up AWS Config. So once you're happy with that and once you're done with that you can click on confirm, and you can see that it's now setting up AWS Config for this region. And that's it. It's set up and it's running. 

So whatever create, deletes, or changes I make to support the resource types within this region AWS Config will not begin to monitor that and record it and log it within the configuration stream and also within the configuration history files. Now what I've done, I've already set this up in another region, and I've made a few changes. So what I'll do now, I'll swap to that region, and we can take a look at some of the other relevants such as the configuration history file, the timeline of events, etc., and look at how the relationships are set up within this dashboard. So let me swap over to another region which is island. 

So as you can see, I've just swapped to the Ireland region, and we don't have the option of setting up AWS Config because I've already set it up and only have it running once within the region. So let's take a look at what we have here. Straight away here we can see that I've had existing rules. I had a rule name of S3 bucket versioning enabled. So I wanted to check if my S3 buckets had versioning enabled. So this is one of the config rules. So if I just open that up I can see that a number of these buckets do not have versioning enabled. And they have been marked as noncompliant. I can still use these buckets. I can still write to the buckets. I can still delete objects from those buckets. It's just notified me that these buckets are noncompliant with this specific config rule, which is to check if, it says it checks whether versioning is enabled for your S3 buckets. 

Now let's have a look at some of the other resources that we have. So I can filter on specific resource type that I want to take a look at. So let me take a look at my VPC. Let me look up everything to do with my VPC. So I can see here that I have two VPCs, and let's take a look at one of them. And this has now taken us to the timeline of events of these VPCs. So on the 15th of March at 11:25 there was a resource discovery. So that's probably when I activated AWS Config within this region and it went out and done a discovery of all resources. And then at 11:32 there was a change on this resource item. So lets take a look at the rest of the page. So under the configuration details we have a number of details here. We have the Amazon Resource Name, the resource type, the ID, the CIDR block of the VPC. So there's different configuration details that we can see for different items. 

Now we can also see the tag name here, which is CA demo and further down we can look at the relationships. So this will show other resources that have a direct relationship with this VPC. So we can see we have a couple of network ACLs there. We have an instant gateway, we've got some root tables, security groups, and some subnets. So each of these resource types has a direct relationship with this VPC, and if I wanted to I can select straight from here to click on that network ACL and it will take me to the details for that ACL. And again, we have some resource type information here, the ARN, and the ID, etc. Again, if we click on the relationships we should see the VPC that we just came from. So let's go back to our VPC. So as you can see it's very easy to look at the relationships between different resources and navigate between them, and it's kind of grouped in a logical manner to allow you to quickly get between the different resource types that you'd like to. 

Now we can see up here that on the 15th of March 11:23 there was a change. Now if we go down to our changes here, we can have a look at what that change was. We can see it's defined by two different categories, a configuration change or a relationship change. We can see that by the number that this change was to do with relationships. And it looks as though a new subnet was added to this VPC. And then finally, we have our CloudTrail events. And this will capture any API calls that made changes to resources. However, it will only capture these for the previous seven days. So because this change was made on the 15th of March and it is now the 30th of March, so if we look at the CloudTrail events there's nothing there, they have expired. So what I'll do, I'll go and make a couple of changes. I'll create a new subnet within this VPC, so I'll pause the video, wait a few minutes, and then I'll start it again and we can analyze the CloudTrail event of that new change. 

Okay, so I've gone off and I've created a new subnet within this VPC so we can see that the 30th of March, which is today, that there was a change that occurred and two new events. So let's go ahead and take a look at the change. So we can see that there is a new subnet. This is the subnet that I just created. And if we look at the CloudTrail events, we can see that we have the creation of the new subnet there. Now if we click on the actual CloudTrail event itself it will take us to CloudTrail and we can look at additional information. So now we're looking at the details of the API call itself, and we can see a number of different information here. We can see the user that created it, the source IP address, the time, the events source, and also the region as well, along with any affected resources as well. 

So here we can drill down into exactly what time it occurred. As you can see up here the date and time and by who and what event actually was created. So as you can see, clicking on that CloudTrail event you can get additional information to kind of help you with resolving instance and looking at potential security breaches to try and gather more information as to identifying who is doing what and when. So now let's take a look at the configuration history, which is stored in S3. So let's go across to S3. So if I go to the bucket that I use for the ireland region, and then navigate through the folders to the correct date and time of today, you can see the configuration history folder. And then if I download one of these files and then open that, we can see here that that configuration history file contained the subnet creation that we just created for our VPC, and it shows you all the different information about it, the availability zone, the CIDR block used, etc., etc. And there's the subnet ID. And it also highlights the relationships to other resources for that new subnet such as any network ACLs that may be associated. So that's the configuration history file that's stored in S3 in a JSON format. That brings us to the end of this demo. So we look to how to set up AWS Config for a region. We then looked at the details of a specific resource type. We looked at the VPC and how the timeline of events work. We looked at a couple of changes and also the CloudTrail event logs as well. And then finally we looked at one of the configuration history files as a JSON document. That now brings me to the end of this lecture covering the AWS Config service and the config history login data.

End of demonstration