Hello and welcome to this lecture where we will be taking a look at the components that make up the AWS Config Service and the functions that each of them carry in delivering the service.

To understand how to get the most of this service, it's important to understand how the service is pieced together and how it works. So let's start by identifying the key components. Following this we will then break each of them down to understand their role and how they sit within the service.

The following identifies the main components to the service. AWS resources, configuration items, configuration streams, configuration history, configuration snapshots, configuration recorder, config rules, resource relationships, SNS topics, S3 buckets and AWS config permissions.

Let's now take a look at each of these in turn in more detail starting with AWS resources. As discussed in the previous lecture, AWS resources are typically classed as objects that can be created, updated or deleted from within the AWS Management Console, or programmatically using the AWS CLI or supported SDK. AWS Config records changes to supported AWS resources within a specific region. For the current list of supported services and resources, please follow the link on the screen.

A configuration item or CI as it's known, is a key component of AWS Config. It is comprised of a JSON file that holds the configuration information, relationship information and other metadata as a point-in-time snapshot view of a supported resource. All the information that AWS Config can record for a resource is captured within the CI. A CI is created every time a supported resource has a change made to its configuration in any way. In addition to recording the details of the affected resource, AWS Config will also record CIs for any directly related resources to ensure the change did not affect those resources too.

For example, if there was a rule change to a security group, additional rules could be added with new ports. AWS Config would record all CI information for that resource, but it would also gather CI information for any instances that were a part of that security group. The CIs would then be sent to a configuration stream, which we will cover next.

As so much data is gathered within these CIs, it's important we look at this in further detail. So for every CI generated, they'll be five different sections.

Firstly, metadata. This essentially contains details about the configuration item itself. So within this metadata, we have both a version ID and a configuration ID, which uniquely identifies the CI. In addition to this, other information can include an MD5Hash that allows you to compare other CIs already recorded against the same resource, as well as ensuring there are no duplications. And then we have the time of the capture and state ID, which puts the CI for a particular resource into an order of sequence, which is useful when multiple CIs for the same resource are sent the configuration stream.

Attributes. This holds common attribute information against the actual resource. Within this section, we also have a unique resource ID, and any key value tags that are associated to the resource. The resource type is also listed. For example, if this was a CI for an EC2 instance, the resource types listed could be the network interface, or the elastic IP address for that EC2 instance. The Amazon resource name, the ARN, for the resource would also be shown along with the availability zone that the resource belonged to. Bear in mind that for services and resources that are not fixed to a particular availability zone, such as IAM and IAM roles, then this section would not be applicable. Lastly, the time the resource was created would also be given.

Relationships. This holds information for any connected relationship that the resource may have. So within this section, it would show a clear description of any relationship to other resources that this resource had. For example, if the CI was for an EC2 instance, the relationship section may show the connection to a VPC along with the subnet that the EC2 instance resides in.

Current configuration. This will display the same information that would be generated if you were to perform a describe or list API call made by the AWS CLI. AWS Config uses the same API calls to get the same information. So depending on the API call and resource, different information will be returned, which is resource specific.

Related events. This relates to AWS CloudTrail. This will display the AWS CloudTrail event ID that is related to the change that triggered the creation of this CI. There is a new CI made for every change made against a resource. As a result, different CloudTrail event IDs will be created. This allows you to deep dive into who, or what, and when made the change that triggered this CI. A great feature allowing for some great analysis to be taken, specifically when this affects security resources.

As you can see, the configuration item is a fundamental aspect of AWS Config when recorded change is made to a supported resource. Because it captures so much data, these CIs are used by other features and components of AWS Config such as configuration history. CIs are used to look up all the changes that have been made to a resource. Configuration streams. CIs are sent to an SNS topic to enable analysis of the data. Configuration Snapshots. CIs are used to create a point-in-time snapshot of all supported resources.

Following CIs, let's move on to configuration streams, which was briefly mentioned a few minutes ago when talking about the configuration items. When a create, update or delete API call is made against a supported AWS Config resource, as we now know, a new CI is created along with additional CIs for any resources that were related to the original modified resource. These CIs are then sent to a configuration stream and this stream is in the form of an SNS topic. However, this stream is also used by AWS Config to send information when other events occur, such as when the configuration history for a resource was delivered to your account, when a configuration snapshot was started and delivered to your account, when the state of your resource compliance changes against any config rules that have been configured, when evaluation begins for rules against resources, and when AWS Config failed to deliver notifications to your account.

This SNS topic can have different notification endpoints, such as email, or an endpoint more commonly used, is SQS, Simple Queue Service. This allows the information to be accessed programmatically to then analyze the data captured. However, you could, like I mentioned, set up your email notification for all stream items, although in a large corporate environment, this may not be ideal as your email inbox would get full up very quickly. If you do decide to do this, then it's worth setting up an email filter to ensure the information you're interested in is made available to you. More information on this configuration can be found on this link here.

This stream for AWS Config should not be confused with the real-time streams that area available through the AWS Kinesis service, which continually capture huge amounts of data from multiple sources such as clickstreams, transactions, social media feeds, and IT logs, etc.

The configuration history uses configuration items to collate and produce a history of changes to a particular resource. This allows you to see the complete set of changes made to a resource over a set period of time. The information can be accessed either programmatically through the AWS CLI using the following command. You can also specify the resource type. So, for example, if you wanted to look at the configuration history for a subnet, you could enter the following into the AWS CLI. Or you could access the history via the AWS Management Console. Additionally, AWS Config also sends a configuration history file for each resource type to a S3 bucket that is selected during the set up of AWS Config. This configuration file is typically delivered every six hours, and like I say, it contains all CI changes for all resources of a particular type.  For example, there would be one configuration history file covering six hours for all RDS DB instance changes in one region.

The configuration snapshot also uses configuration items in its production. The configuration snapshot will take a point-in-time snapshot of all supported resources configured for that region. It will generate CIs for each resource in your AWS account for a specific region, and this configuration snapshot can then be sent to an S3 bucket. Alternatively, this information can be viewed via the AWS Management Console.

Now let's take a look at the configuration recorder. So this component of AWS Config can be seen as the engine of the service. This element is responsible for recording all of the changes to the supported resources within your account and generating the configuration items. By default, the configuration recorder is automatically enabled and started when you first configure AWS Config. However, it is something that you can stop and then restart again at a later point. When you stop it, AWS Config will no longer track and record changes to your supported resources.

When you first configure AWS Config in the AWS Management Console, you are asked which resource types you would like to record. This is essentially setting up and creating the configuration recorder, as this information is required to enable the configuration recorder to start. If you only select to record specific resource types, then it will capture all creates, changes, and deletes for those resource types and will create a CI record for each occurrence. However, the configuration recorder will still record all creates and deletes of supported resource types within that region, but no changes to those resources will be recorded. Also, if you stop and change your configuration recorder settings to perhaps remove certain resource types from being recorded, then all captured information for those resources up to that point will remain and you will be able to view their history with all data from the previous CIs that had been captured.

AWS config rules are a great way to help you enforce specific compliance checks and controls across your resources, and allows you to adopt an ideal deployment specification for each of your resource types. Each rule is essentially a lambda function that when called upon evaluates the resource and carries out some simple logic to determine the compliance result with the rule. Each time a change is made to one of your supported resources, AWS Config will check the compliance against any config rules that you have in place. If there was a violation against these rules, then AWS Config will send a message to the configuration stream via SNS and the resource will be marked as non-compliant. It's important to note that this does not mean the resource will be taken out of service or it will stop working. It will continue to operate exactly as it is with its new configuration.

AWS Config simply alerts you that there is a violation, and it's up to you to take the appropriate action. These rules can be custom defined, or selected from a predefined list of AWS managed rules that AWS has created on your behalf. Being able to create your own rules allows you to adopt best practices that you may have internally within your own enterprise or with other security best practices. By allowing AWS Config to monitor resources at this level, it adds another level of automation, helping to prevent misconfigurations made by human error being left, which could lead to a security risk, or even worse, a breach. I highly recommend using config rules for maintaining security checks and configurations.

AWS have a number of predefined rules that fall under the security umbrella that are ready to use. For example, Rds-storage-encrypted. This checks whether storage encryption is activated by your RDS database instances. Encrypted-volumes. This checks to see if any EBS volumes that have an attached state are encrypted. Optionally you can specify the ID of a KMS key to use to encrypt the volume. Rootaccount-mfa-enabled. This checks whether your root account of your AWS account requires multifactor authentication for console sign in. IAM-user-no-policy-check. This checks that none of your IAM users have policies attached. Best practice dictates that permissions should be provided via roles or groups.

The great thing about these predefined rules is that you can also edit them to make subtle parameter changes as needed. As you can see, being able to ask AWS Config to check your resource's compliance with rules such as these are invaluable. And when you couple this with being able to create your own custom rules, the scope and potential of automating compliance is huge across your supported resources.

When you first use AWS Config with a set of rules, whether they are predefined or custom, don't be surprised by the results that you find. It's likely that you will see more resources being flagged as non-compliant than you may have imagined. As I have already mentioned, these config rules are great as a security tool to help you mitigate potential security issues that may arise within your environment. These config rules are also a great way to help you meet specific compliance and governance controls for auditing. You can currently use 50 config rules per region. However, if you are finding that you require more, then you can request an increase via AWS.

Coming up later in this course, we shall have a demonstration on how to set up custom config rules where I shall talk more about the different options available.

As a part of the CI for a particular resource, for example, an EBS volume, AWS Config identifies relationships with other resources from that resource. In this case, it might be the EC2 instance that the volume is attached to. This allows AWS Config to build a logical mapping of resources and how they connect. This mapping of relationships allows you to quickly jump to other linked resources within the AWS Management Console to view their configuration history and CI data. This is very useful if you're trying to troubleshoot an issue and pinpoint where the source of an incident may be. For a full listing of available relationship types, see the link on the screen.

As we have already seen, an SNS topic is used as a configuration stream for notifications of various events triggered by AWS Config. You can have various endpoints associated to the SNS stream. Best practice indicates that you should use SQS and then programmatically analyze the results via SQS. The S3 bucket that was selected at the time of configuration is used to store all the configuration history files that are generated for each resource type, which happens every six hours. Also any configuration snapshots that are taken are also stored within the same S3 bucket. The configuration details used for both SNS and S3 are classed as the AWS Config delivery channel by which data can be sent to other services.

When setting up AWS Config, you're required to select an IAM role. This role is required to allow AWS Config to obtain the correct permissions to carry out and perform a number of functions. For example, AWS Config will need read-only access to all the supported resources within your account so it can retrieve data for the configuration items. Also, we now know that AWS Config uses SNS and S3 both for streams and storage of the configuration history files and snapshots. So AWS Config requires the relevant permission to allow it to send data to these services.

That now covers the key elements of AWS Config and so I hope it makes it clear as to how each part plays a role within the service. To reiterate, let's take a bulleted point look at how it all comes together. When you first turn on AWS Config, you will configure the elements required for the configuration recorder to begin capturing and recording data. AWS Config will then discover all your supported resources based upon the details entered within the configuration recorder within that region. When a create, change or delete against a resource is made, a CI will be created for this change and a notification will be sent to the configuration stream regarding the new CI. Also, following the CI, AWS Config will check its current config rules to evaluate if the change has made a resource non-compliant. If the evaluated state changed for the resource, a notification will be sent to the configuration stream. Your config rules can also be configured to run periodically, and so they will run at a set given time period regardless if there have been any changes to resources. If a configuration snapshot is taken, AWS Config will create a point-in-time snapshot of the resources with new CIs, and deliver this configuration to your specified S3 bucket within the configuration recorder. After six hours has passed from turning on AWS Config, a configuration history file will be created for each resource type, and again sent to your specified S3 bucket. The configuration file will be sent every six hours from that point.

Now we have learned the theory behind AWS Config, let me introduce you to the service with a demonstration. In this demonstration, I will run through how to set up AWS Config, pointing out components such as the configuration recorder along the way. We will also look at the AWS Config dashboard showing the timeline of events, and how by using relationships of your resources, you can navigate through your infrastructure. During this demo, I'll also show you the configuration history file within S3.

Okay, so I've just logged into my AWS account, and I'm going go up to Services and you can see that AWS Config is under Management Tools. So if we select Config. And this is the first splash screen you'll be presented with if you don't have AWS Config started as yet. So, let's get started by clicking on the Get Started button, and then there's three steps to setting AWS Config up.

So the first step is configuring the settings, and as I mentioned earlier in the lecture, we have our resource types to record and this is where we essentially start the configuration recorder by identifying what resources we'd like included in our AWS Config configuration. So, starting from the top, we can either select all resources within this region. Remember AWS Config is region specific, so we can choose to record all the resources supported within this region, and currently I'm in the London region. And we can choose to include global services, like we also discussed, services such as IAM, etc. But for this demonstration, we'll just choose to record all resources in this region. Here you can select specific types if you require, so if I untick this, we can then go down to this drop-down box and select specific resources types that we'd like set up for this configuration. So you can be quite specific, selecting the load balancer or different elements of IAM, and again, different element of RDS, etc. So, there's a list there that you can go through if you just want to record a specific resource type, but like I said, for this demonstration, I'm just going to select all resources.

Next we need to define our S3 bucket, and this is where our configuration history and snapshot files are stored. So we can ask AWS Config to just create a new bucket for us, and it'll prefix it with this bucket name here. Or we can choose an existing bucket from my account by selecting the drop-down list, or we can select a bucket from another account. So if you had multiple accounts, you can send the configuration history files and snapshot files for all those accounts to a single, primary AWS account in a single bucket. So for this example, though, for this demonstration, I'm just going ask AWS Config to create a new bucket for us. Config bucket cloudacademy.

Now I'm moving down to the SNS topic. Now this is the configuration stream, as we discussed, so for any events that AWS Config picks up, it will send it to this SNS topic for the configuration stream. Now we can create a new topic, and again, AWS Config will just give us a topic name here, or we can select to choose another topic from your account or again from another account. For this demonstration, we'll just ask AWS Config to create the topic name for us. And I'll just add in cloudacademy.

Now, finally, we have the AWS Config role, and this is where we spoke about the different permissions. And this is required to allow AWS Config to send theconfiguration history file and snapshots to S3 as well as having access to send events to the SNS topic. And not forgetting enabling AWS Config to be able to kind of go out and poll your resources as a describe or list API call to get the details of those resources. And again, for this demonstration, I'm just going to ask AWS Config to create a role for us. I'll just add cloudacademy on the end.

So this screen is essentially your configuration recorder. By setting this up, you're asking AWS Config to start recording. We have set up the S3 bucket to allow our configuration history files and our snapshot files to be captured. And we have created our configuration stream by configuring an SNS topic. And we have also allowed AWS Config to have the necessary permissions to carry out those functions. So once that is all set up, we click on Next.

And here we have a set of predefined AWS managed config rules. And, like I said earlier, this is a number of templates that you can select to check the compliance of you resources against these rules. We'll cover more on AWS Config rules in a later lecture and we'll also have a demonstration on how to create one and modify one of these existing AWS managed rules. So for now, I'm just going to click on Skip.

And here we're on the final stage, which is review. So we can see here the resource types we've selected, which are all resources, and we've not chosen to include global resources. We have our S3 bucket set up for our history files and snapshots. And we have our SNS topic for our configuration stream configured, and we have the permissions given by the AWS Config rule. And that's essentially the elements to setting up AWS Config. So once you're happy with that, and once you're done with that, you can click on Confirm, and you can see that it's now setting up AWS Config for this region.

And that's it. It's set up and it's running. So whatever create, deletes or changes are made to supported resource types within this region, AWS Config will now begin to monitor that and record it and log it within the configuration stream, and also within the configuration history files.

Now what I've done, I've already set this up in another region and I've made a few changes. So what I'll do now, I'll swap to that region and we can take a look at some of the other elements, such as the configuration history file, the timeline of events, etc., and look at how the relationships are set up within this dashboard. So let me swap over to another region, which is Ireland.

So as you can see, I've just swapped to the Ireland region and we don't have the option of setting up AWS Config because I've already set it up and you only have it running once within the region. So let's take a look at what we have here. Straight away we can see that I've had existing rules. I had a rule name of s3-Bucket versioning-enabled. So I wanted to check if my S3 buckets had versioning enabled. So this is one of the config rules. So if I just open that up, I can see that a number of these buckets do not have versioning enabled, and they have been marked as non-compliant. Now remember, I can still use these buckets. I can still write to the buckets. I can still delete objects from those buckets. It's just notified me that these buckets are non-compliant with this specific config rule, which is to check if... Right, it says here, it checks whether versioning is enabled for your S3 buckets.

Now let's have a look at some of the other resources that we have. So I can filter on a specific resource type that I want to take a look at. So let me take a look at my VPC. Let me look at everything to do with my VPC. So I can see here that I have two VPCs, and let's take a look at one of them. And this is now taking us to the timeline of events of these VPCs. So on the 15th of March at 11:25, there was a resource discovery. So that's probably when I activated AWS Config within this region and it went out and done a discovery of all resources. And then at 11:32, there was a change on this resource item. So let's take a look at the rest of the page.

So under the configuration details, we have a number of details here. We have the Amazon resource name, the resource type, the ID... The CIDR block of the VPC. So, there's different configuration details that we can see for different items. Now we can also see the tag name here, which is CA-Demo, and further down, we can look at the relationships.

So this will show other resources that have a direct relationship with this VPC. So we can see we have a couple of network ACLs there, we have an internet gateway, we have some root tables, security groups and some subnets. So each of these resource types has a direct relationship with this VPC. And if I wanted to, I can select straight from here to click on that network ACL, and it will take me to the details for that ACL. And again, we have some resource type information here, the ARN and the ID, etc. Again, if we click on the relationships, we should see the VPC that we just come from. So let's go back to our VPC.

So as you can see, it's very easy to look at the relationships between different resources, and navigate between them. And it's kind of grouped in a logical manner to allow you to quickly get between the different resource types that you'd like to. Now we can see up here that on the 15th of March, 11:23, there was a change. Now if we go down to our changes here, we can have a look at what that change was. We can see it's defined by two different categories, a configuration change or a relationship change. We can see that by the number, that this change was to do with relationships. And it looks as though a new subnet was added to this VPC.

And then finally, we have our CloudTrail events, and this will capture any API calls that made changes to resources. However, it will only capture these for the previous seven days. So, because this change was made on the 15th of March, and it's now the 30th of March, so if we look at the CloudTrail events, there's nothing there. They have expired. So what I'll do, I'll go and make a couple of changes. I'll create a new subnet within this VPC. So I'll pause the video, wait a few minutes and then I'll start it again and we can analyze the CloudTrail event of that new change.

Okay, so I've gone off and I've created a new subnet within this VPC. So we can see that the 30th of March, which is today, that there was a change that occurred and two new events. So let's go ahead and take a look at the change. So we can see that there is a new subnet. This is the subnet that I just created. And if we look at the CloudTrail events, we can see that we have the creation of the new subnet there. Now if we click on the actual CloudTrail event itself, it will take us to CloudTrail and we can look at additional information.

So now we're looking at the details of the API call itself, and we can see a number of different information here. We can see the user that created it, the source IP address, the time, the event source and also the region as well, along with any affected resources as well. So here we can drill down into exactly what time it occurred. As you can see up here, the date and time, and by who, and what event actually was created. So, as you can see, clicking on that CloudTrail event, you can get additional information to kind of help you with resolving incidents, and looking at potential security breaches to try and gather more information as to identifying who is doing what and when.

So now let's take a look at the configuration history, which is stored in S3. So let's go across to S3. So if I go to the bucket that I use for Ireland region, and then navigate through the folders to the correct date and time of today, I can see the configuration history folder. And then if I download one of these files, and then open that. We can see here that that configuration history file contained the subnet creation that we just created for our VPC, and it shows you all the different information about it, the availability zone, the CIDR block used, etc., etc. And there's the subnet ID. And it also highlights the relationships to other resources for that new subnet such as any network ACLs that may be associated. So that's the configuration history file that's stored in S3 in a JSON format.

That brings us to the end of this demo. So we looked at how to set up AWS Config for a region. We then looked at the details of a specific resource type, we looked at the VPC and how the timeline of events work. We looked at a couple of changes, and also the CloudTrail event logs as well. And then finally, we looked at one of the configuration history files as a JSON document. That brings us to the end of this lecture. Coming up next, we will look at how AWS Config integrates with other AWS services in a bit more detail.