The course is part of these learning paths
About AWS CloudTrail
AWS CloudTrail Configuration
Any information that helps to secure your Cloud infrastructure is of significant use to security engineers and architects, with AWS CloudTrail you have the ability to capture all AWS API calls made by users and/or services.
Whenever an API request is made within your environment AWS CloudTrail can track that request with a host of metadata and record it in a Log which is then sent to AWS S3 for storage allowing your to view historical data of your API calls.
Having this information has a number of uses from a security and day to day operational perspective, but it also allows for additional compliance and when it comes to specific security governance controls. Having an audited trail of requests which can be tracked backed to a user or service, and even the IP address used helps to maintain your required compliance levels.
This course provides a full explanation of the service, looking at what it does, how it does it and with what components and services. It breaks down each of the configurable components allowing you to see exactly how it works and to what degree it can be configured.
It dives into permissions required to run and implement CloudTrail, covering roles and policies, along with an overview of S3 Bucket permissions required for log storage. There are also a number of demonstrations within the course showing first hand how to configure Trails and set up various controls and permissions giving you clear guidance on what to do.
CloudTrail Logs are examined to show you exactly how APIs are recorded and how this sensitive information can be encrypted using KMS and also shared between AWS Accounts.
A key aspect of CloudTrail is its relationship with AWS CloudWatch, where the two services can be used together to create a monitoring solution based on API requests, allowing custom metrics and thresholds to be created. When used in conjunction with SNS, this becomes a powerful monitoring solution.
If you have thoughts or suggestions for this course, please contact Cloud Academy at email@example.com.
Hello, and welcome to this lecture where we will explain CloudTrail trails and their configuration in further detail.
We learned earlier that Trails are an essential prerequisite for making use of the benefits of CloudTrail. Simply put, without a Trail, the service is unable to capture API calls. We also learned that these Trails are the foundation building blocks that hold all configuration information for isolating which API calls are to be recorded. So let's start at the beginning. Creating a Trail via demonstration. I'll talk through how we create a Trail from within the AWS Management Console. Once we're at the CloudTrail dashboard, I will then create a Trail and explain the different components to configure in each section. Let's get started.
Okay, so let's take a look at how to set up a Trail within CloudTrail. As you can see, I'm currently at the Service page within the AWS Management Console. And if we go across to Management Tools, and go down to CloudTrail, that's where you'll find the service. So if we click on that. And that will take us to the dashboard of the CloudTrail service. And if we click on Trails, here is the screen where you can add new Trails and see all your existing Trails that you already have configured.
So, for this demonstration, let's add a new Trail. The first thing one needs to do is add a Trail name. So, let me label this CloudAcademy Trail. We next need to decide if we want to apply this Trail to all regions. It's either yes or no. If we select no, then it will apply the Trail to the current region that you have configured within your Management console. For this demonstration though, I'm going to apply the Trail to all regions.
Next, we're asked to create a new S3 bucket. So, the S3 bucket is the destination of where your log files will be stored, so all the events that are captured that relate to API calls are recorded into logs, and then these logs are delivered to an S3 bucket. So here we can either create a new S3 bucket or say no and select an existing bucket that we have. For this demonstration, I'm going to say yes, and by doing so, CloudTrail will apply all the correct bucket policy permissions to allow CloudTrail to deliver logs to that bucket. So, let's give this bucket a new name. I'll call it CloudAcademy Trail. Keep it the same name.
And if I click on Advanced, we're then presented with a host of other options. Firstly is a log file prefix, and if we click on the information button here, you can see that a prefix makes log files easier to browse. And it does this by creating a folder-like structure for your prefix. So let's give it a prefix of demo. So, as you can see here, the location is given, the prefix is demo, kind of a separate folder underneath your S3 bucket.
Our next option is to encrypt log files. By default, CloudTrail logs are stored on S3 with server-side encryption. So it's automatically encrypted. However, if you wanted to add an additional layer of encryption, you can say yes to this and select to use KMS, which is Key Management Service. So we can create a new Key Management Service here or use an existing one and select an existing key that we may have. Now, I won't go into this hugely right now, because I do cover KMS encryption later on in the course. But I just wanted to point out at this stage that if you did want to apply additional security to your logs, then you can select yes to encrypt log files here using KMS. But for this demonstration, we'll leave that as no.
Now, next on the configuration options is log file validation. We can either enable this as yes or no. Now what log file validation does, it will check to ensure that your log files haven't been altered, modified, or tampered with in any way, once they've been delivered to your S3 bucket. Now, this is typically used for security and forensic investigations or to be compliant with certain security and governance controls. For this demonstration, I'm going to select no for log file validation. And again, later on in this course, I do go deeper into this topic.
Next, we have a notification option using SNS, so Simple Notification Service, that will notify us when a new log file is delivered to the bucket. If I click on yes here, then we can create a new SNS topic or select an existing topic. So if we did have this enabled, then any recipients associated to the topic will be notified every time a new log file is delivered. For this demonstration, I'm going to select no for that. And that's it. So, to create your Trail, it's just these few options and configuration details that you need. So now I've selected all those. I'm going to click on Create.
And there you can see in my list of Trails, you can see the new name, CloudAcademy Trail. And we specified that it was all regions, gave it a bucket name, a log file prefix of demo, we haven't associated any CloudWatch logs yet, and the logging status is on. So, although we've created this new Trail, there are additional configuration options we can apply. But we can only do that once the Trail is created. So to add that additional configuration, such as adding the option to send the logs to CloudWatch, we'll need to select the Trail here first.
And this opens up another page with additional configurable options on it. So, let's run through these from the top. So, the first option here is Apply trail to all regions. And we already indicated yes on the previous screen. However, we didn't have this option of Include global services. So what does this mean? Global services, such as IAM and CloudFront, aren't region-specific. So, when events are triggered by these, those events are sent to any Trails that include global services. So, where this says yes. Most services are region-specific, so any events triggered from these are sent to the region that they were made from. It's also worth pointing out as well that global services are management events, so if your event selectors within your Trail do not include management events, then these global services will not be logged. But I'll cover event selectors as we go through the rest of this configuration.
As we scroll down to the S3 section, we can see all of the details that we added earlier, like the S3 bucket name, the prefix, whether or not to encrypt the log files, etc, etc.
Now, here on the next section is where we can configure CloudWatch logs. Now, this is an optional component, so what this enables you to do is to send your CloudTrail logs onto CloudWatch. And from there, you're able to set up specific metrics and monitoring to allow you to monitor for specific events within your CloudTrail logs. So any event that's recorded, you can set up monitoring for. And if you can monitor on that, then you can also set up an SNS alert against that monitoring when it meets certain thresholds that you have configured. I just want to be clear here, though, that just because we're sending it to CloudWatch, it also sends it to S3. So the logs are delivered to both the S3 bucket and also the CloudWatch logs to allow you to set up monitoring.
Now, if we go into Configure, we can enter a new or existing log group within CloudWatch. If you're setting up a new log group, then as a part of this process, CloudTrail will set up all the necessary permissions and policies and roles as well, because for this to work, CloudTrail needs to assume a role to carry out a couple of API calls itself. But I'll cover this in more detail in a later lecture. But just do be aware that if you want to configure CloudWatch logs, then it's done in this section here.
Before I move on, though, I will click on Continue to quickly show you the roles, etc, that it uses. And you can see here that, in order to successfully deliver CloudTrail events to your CloudWatch logs group, CloudTrail will need to assume a role, and that is for these two API calls here. If you look at the details, we can select the IAM role and create a new role or an existing, and then a policy to go around that role as well. But like I said, I'll go into this in more detail in a later lecture. We'll cancel that.
Now we come down to event selectors. Again, this is an optional component. And event selectors specify the types of events that CloudTrail logs. Now, as you can see, there's three different types of event selectors. There's the Read/Write events, Management events, and Data events.
If we go in to edit these, it gives us a couple more options. So, for the Read/Write events, we can select either Read-only, Write-only, or all. So, this, remember, this is referring to what events this Trail will capture and write to a log. So any event that happens in our account, CloudTrail will look at all of the Trails and the event selectors, and if that event matches any event selectors within a Trail, then it will get written to the log of that Trail. So, for this particular Trail, we can have Read-only, Write-only, or All. For this example, I'm going to leave it on All, and this Read/Write events applies to both Management events and Data events.
If I go down to Data events first. I'll come back to Management events in a moment. We'll go down to Data events first. So, Data events are object-level API calls to S3 objects, such as put, get, and delete object. Everything else is classed as a Management event. So, all other API calls fall under the Management event category. So, for the Data events, you would need to enter a bucket name and a prefix, if required, and then, if you have that enabled, then any object-level API calls to objects within that bucket will be logged as an event within your CloudTrail log.
And like I said, all other events are classed as Management events. You can either have that on or off. If we switch off Management events, then we need to have a Data event configured, because otherwise, if we switch off Management events and don't have any Data events, then effectively this Trail has no event selectors to log. So, I'll switch the Management events back to yes.
If we go down to Add new event selector, we can see here that we can add multiple events to the same Trail. So, for example, I can have this as Write-only and switch off Management events and then have a Data event that looks at our CloudAcademy Trail bucket. So, if we look at this first event selector, we can see that any read or write event that happens within our account, that is classed as a Management event, and it will be written to the logs. And also, if we have a write-only event, that falls under a Data event within this S3 bucket, so any kind of delete or put event that happens within this bucket will be recorded to the log as well. But I'm just going to cancel out of that. And leave as the default Management Yes, and the Data events No.
So, coming down to our last configurable item, which is Tags, which again, is optional. If we go into Edit, we can add a unique key value pair here, so, perhaps this relates to a particular project. And we can save our Trail project name of Security Course, or any other tag that's relevant to your solution. And then simply click on Apply.
And that's it. That is your CloudTrail configured. And just before I finish this demo, you can see that at the very top here, you can switch your CloudTrail logging on and off. So I can switch that off, and it will say you no longer collect log files in your S3 bucket or your CloudWatch log group. Say Continue. And your login is, from that point, turned off. Now, if we click on Trails, and there again, we can see that our CloudAcademy Trail, the login status is now off. And that's how you create a new Trail in CloudTrail.
About the Author
Stuart has been working within the IT industry for two decades covering a huge range of topic areas and technologies, from data centre and network infrastructure design, to cloud architecture and implementation.
To date, Stuart has created 50+ courses relating to Cloud, most within the AWS category with a heavy focus on security and compliance
He is AWS certified and accredited in addition to being a published author covering topics across the AWS landscape.
In January 2016 Stuart was awarded ‘Expert of the Year Award 2015’ from Experts Exchange for his knowledge share within cloud services to the community.
Stuart enjoys writing about cloud technologies and you will find many of his articles within our blog pages.