Amazon Macie Configuration
Amazon Macie was launched in the summer of 2017, much to the delight of cloud security engineers. Amazon Macie is a powerful security and compliance service that provides an automatic method to detect, identify, and classify data within your AWS account. Macie currently supports Amazon S3 storage, however additional support for other storage systems will be developed and added over time. Backed by machine learning, Macie can actively review your data as different actions are taken within your AWS account. Machine learning spots access patterns and analyzes user behaviour using CloudTrail event data to alert against any unusual or irregular activity. Any findings are presented within a dashboard which can trigger alerts allowing you to quickly resolve any potential threat of exposure or compromise to your data.
This course will dive into all elements of the service, discussing its many different features and customizable elements allowing you to gain the maximum potential of its ability.
By the end of this course you will be able to:
- Provide an understanding and awareness of what Amazon Macie is and what it’s used for
- Provide an explanation of each configurable component of the service to allow you to gain maximum benefit from Macie’s capabilities
- Understand how the service can provide a customizable approach to maintaining compliance
- Understand how through automation and machine learning Amazon Mazie detects and categorizes S3 content to detect potential security threats and exposures
The content of this course is centered around security and compliance. As a result, this course is beneficial to those who are in the roles or their equivalent of:
- Cloud Security Architects
- Compliance Managers
- Cloud Administrators
- Cloud Support & Operations
As a prerequisite of this course you should have an understanding and awareness of:
- Amazon S3
- AWS CloudTrail
Course: AWS Cloudtrail: An Introduction
Hello and welcome to this lecture. Now we have an understanding of what Amazon Macie is, let me now explain how to enable it so you can associate it to your Amazon S3 data to understand any potential security issues.
As I mentioned previously, the services located under the security identity and compliance category within the AWS Management Console. When you first go into the service for the first time you'll be presented with a splash screen similar to the following. From here you simply click on get started. Now at this point Amazon Macie will check your AWS account for specific requirements that are needed before you can go ahead and fully enable Amazon Macie. These requirements are to check the existence of IAM roles, specifically the AWSMacieServiceCustomerSetupRole and to check if AWS CloudTrail is enabled within your AWS account. Both of these are prerequisites to being able to enable Amazon Macie. If these components are not configured within your AWS account you will see the screen like this. As you can see, there are two red check marks against each of these requirements.
To resolve the first issue of the IAM roles, you will need to launch a preconfigured AWS CloudFormation Stack that has been created by AWS that will automatically set up and configure the roles that are needed. Dependent on your region that you're using will depend on the CloudFormation Stack used. Currently there are templates for US East Virginia and US West Oregon. The URL links to these stacks can be found in the transcript of this lecture along with the URL source of these templates, allowing you to find more regional stacks as and when they are released.
Before I continue I just want to quickly mention that if you'd like additional information on AWS CloudFormation, then you can view our existing content here. We have a couple of courses, how to use CloudFormation for AWS Automation and Advanced Use of AWS CloudFormation, and we also have a lab Deploying Wordpress using AWS CloudFormation.
Okay, back to our CloudFormation Stacks. Regardless of which one you use dependent on your region, the process is very simple. So don't worry if you're not familiar with CloudFormation. When you click on one of the links for the stacks it will open up CloudFormation within your AWS account, providing you are logged into your account already. The data and information will already be prefilled and all you need to do is to accept the defaults. On the select template page you will see that it has already preconfigured the relevant template under the specify an Amazon S3 template URL heading. To proceed to the next screen press next. At the specified details screen leave the default stack name of MacieServiceRolesMaster and click next. On the options screen leave all settings as default and click next. On the final review screen you will need to acknowledge the message that CloudFormation might create IAM resources with custom names via checkbox. Once you have done so click create. At this point the CloudFormation Stack will be created and will generate the required resources defined by the stack, which includes the necessary IAM roles and policies that Amazon Macie requires.
The next requirement needed by Amazon Macie is the enablement of AWS CloudTrail. If you are not familiar with CloudTrail then you can see our existing content of the service here. We have a course, AWS CloudTrail: An Introduction and a lab, Monitoring AWS CloudTrail events with Amazon CloudWatch. The following demonstration will explain and show how to create and enable a new trail within CloudTrail to fulfill this second requirement.
Okay, so I'm logged into my AWS account and CloudTrail is under management tools here. So if we just select CloudTrail, and that will take us to the dashboard. And now from the dashboard on the left-hand side we can then create a trail here, which is what we need to do. So let's create a new trail. We'll call this trail name Macie-demo, and we can apply this trail to all regions. And just leave that as a default yes. Under management events, we want to be notified of all read/write events.
If we scroll down to data events here you can see S3 and lambda. We're only interested in S3 so let's select all S3 buckets in your account. And by default it selected all read and write actions. If we wanted just to do specific buckets then we could add the bucket here, but we've said we will select all three S3 buckets in our account. Under storage location this is where it'll store the CloudTrail logs, and we can create a new S3 bucket for this. So let's give this a name of CloudTrail logs. And if we go down to advanced, here we can add a log file prefix if we wanted to, but there's no need to for this demonstration. We can crypt our log files if we wanted to, and we can select a KMS key. For this demonstration I'm just going to leave that as a default of no.
Log file validation determines if the log file has been tampered with, so you can either have that on or off, and also here you can select to have an SNS notification for every long file delivery. And again, I'm going to set the default for no. So once that's been selected you can click on create. As I should have expected, this bucket order exists. So I'm going to add cloud academy to the end of that because it needs to be a unique bucket name, as we know. That will then go ahead and create the trail. And here we have the new trail and the status is running. And that's it. Now that we have both requirements completed for Amazon Macie we will now be able to enable the service. As you can see I now have both requirements fulfilled indicated by tick. There is one final element to address and that is the permissions checkbox that you can see at the bottom of the page. This simply asks you to acknowledge that through the enablement of Amazon Macie you are happy that the service will have permission to analyze you AWS CloudTrail logs and events. It's worth noting that you can always disable Amazon Macie at any point, and this will in turn stop the monitoring and analysis of your logs. When you select the tick box you will then be able to enable Macie.
Once Macie is enabled you will be taken to the console of the service, which will look something like this. From here our next step is to associate our Amazon S3 buckets that we want Amazon Macie to monitor. The best way to show you how to do this is via a quick demonstration.
Okay, so I'm back in my AWS account, and I've just opened up Amazon Macie. From here what I need to do is on the left-hand side go down to integrations. And then you can see at the top here accounts and services. What I need to select is services. Now I need to select the account I want to integrate other services with Amazon Macie, and this is my AWS account. And at the moment the only viable option is Amazon S3. Overtime there will be more and more storage services added to this section. So all I need to do is click on add. And now here it's asking me which S3 buckets I want Macie to monitor. So I've created a bucket down here called macie-demo-cloudacademy. So I just want Amazon Macie to monitor this bucket here. And I also want Macie to classify all the data within this bucket. And by classifying all the content within that bucket, as I add more and more content to this bucket then Amazon S3 will automatically detect and classify all that data within it to notify me of any potential issues. Once I've selected that I then click on review and save.
There's a couple of tick boxes that you need to select here, the first one to say that you understand that S3 object-level logging is enabled for all buckets and that you understand that choosing to classify all objects in the selected S3 bucket can significantly affect the content classification costs. And then you click on save. And that's it. Now your bucket will be classified by Amazon Macie, and it will be monitored as well. So it's a very simple quick and easy process. We now have our Amazon Macie account enabled and our Amazon S3 bucket's being monitored, which brings me to the end of this lecture.
Over the next few lectures I'll be explaining the different elements of Amazon Macie console to help us understand how we configure and utilize it effectively as a compliance and security tool, starting with alerts.
Stuart has been working within the IT industry for two decades covering a huge range of topic areas and technologies, from data center and network infrastructure design, to cloud architecture and implementation.
To date, Stuart has created 150+ courses relating to Cloud reaching over 180,000 students, mostly within the AWS category and with a heavy focus on security and compliance.
Stuart is a member of the AWS Community Builders Program for his contributions towards AWS.
He is AWS certified and accredited in addition to being a published author covering topics across the AWS landscape.
In January 2016 Stuart was awarded ‘Expert of the Year Award 2015’ from Experts Exchange for his knowledge share within cloud services to the community.
Stuart enjoys writing about cloud technologies and you will find many of his articles within our blog pages.