Amazon Macie Configuration
Amazon Macie was launched in the summer of 2017, much to the delight of cloud security engineers. Amazon Macie is a powerful security and compliance service that provides an automatic method to detect, identify, and classify data within your AWS account. Macie currently supports Amazon S3 storage, however additional support for other storage systems will be developed and added over time. Backed by machine learning, Macie can actively review your data as different actions are taken within your AWS account. Machine learning spots access patterns and analyzes user behaviour using CloudTrail event data to alert against any unusual or irregular activity. Any findings are presented within a dashboard which can trigger alerts allowing you to quickly resolve any potential threat of exposure or compromise to your data.
This course will dive into all elements of the service, discussing its many different features and customizable elements allowing you to gain the maximum potential of its ability.
By the end of this course you will be able to:
- Provide an understanding and awareness of what Amazon Macie is and what it’s used for
- Provide an explanation of each configurable component of the service to allow you to gain maximum benefit from Macie’s capabilities
- Understand how the service can provide a customizable approach to maintaining compliance
- Understand how through automation and machine learning Amazon Mazie detects and categorizes S3 content to detect potential security threats and exposures
The content of this course is centered around security and compliance. As a result, this course is beneficial to those who are in the roles or their equivalent of:
- Cloud Security Architects
- Compliance Managers
- Cloud Administrators
- Cloud Support & Operations
As a prerequisite of this course you should have an understanding and awareness of:
- Amazon S3
- AWS CloudTrail
Hello, and welcome to this final lecture within this course covering Amazon Macie. Hopefully you should now have a better understanding of Amazon Macie, and how it can be used to enhance your security level within your AWS account. Specifically across your data being stored in Amazon S3. The abilities of being able to automatically classify data and its potential risk value, and being able to identify security loopholes and potential exposures with your business critical data is invaluable. Amazon Macie provides a means of ensuring you have a way of enabling compliance on different levels. For example, it can be used as a service to help you enable compliance regulations to be met by GDPR. Through the use of alerts, metrics, deep analysis, best practices and customization, you can use Amazon Macie to meet stringent compliance needs within your business.
I now want to review and highlight some of the key points taken from each of the lectures within this course. I started by focusing on what Amazon Macie was and in this lecture we learnt that the main function of this service is to provide an automatic method of detecting, identifying, and also classifying data that you are storing within your AWS account. It's backed by machine learning to detect access patterns and look for unusual or irregular activity. Findings are presented within a dashboard which can trigger alerts, and Amazon Macie, automatically and continuously monitors data in S3. Natural language processing methods are used to help classify data types and content. Objects are assigned a risk value based on data classification. Amazon Macie can monitor and discover security changes governing your data, and it can detect sensitive and security focused data, such as API keys, secret access keys, in addition to PII and PHI data. It can also detect changes to security policies and access control lists, and alert against unusual user behavior . This all helps you to maintain compliance requirements as needed.
Following this lecture, I looked at how to enable the service and associate your S3 data. In this lecture I explained that your AWS account needs to meet two requirements before you can enable Macie. You need to check the existence of IAM roles, specifically the AWS Macie service customer setup role. And to check that AWS CloudTrail is enabled within your AWS account. Point one here can be implemented through the use of CloudFormation templates provided by AWS. And point two simply requires that you create a trail within CloudTrail. When both requirements are met, you can then enable the service. You can then associate your Amazon S3 buckets with Macie via the integrations menu in the console. During this particular lecture, I provided a demonstration on how to do this.
Next I focused my attention on the different types of alerts generated and that are available with Amazon Macie. By default, Macie is pre-configured with a wide range of alerts based on security best practices and the sensitivity of data that the service will check against. Macie offers the ability to create custom alerts. These alerts exist as two different types. Basic, which consist of prebuilt alerts that come with Amazon Macie, and also custom alerts. Predictive alerts look at the behavior of your AWS account to automatically identify activities that sit outside the realms of normal operations. Alerts displayed in the console show summarized details, however these details can be expanded by clicking on the alert itself. The alert summary shows information allowing you to respond to the alert appropriately with the findings given. The alert detail section offers a whole host of additional information retrieved by CloudTrail events. It's possible to whitelist users for specific alerts that are identified. The severity of an alert can either be informational, low, medium, high, or critical. I also provided a demonstration on how to create your own alerts.
Following this lecture, I discussed the Amazon Macie dashboard. In this lecture, we learnt that the Amazon Macie dashboard is the central hub of information that is collated, monitored and classified through Amazon CloudTrail logs and any services associated to Macie, such as Amazon S3. The dashboard has four metric boxes at the top of the page. Critical assets, this metric defines as a percentage how many of your assets have been identified as high risk, which is anything with a risk value of eight, nine, or 10. Total event occurrences metric. This relates to your Amazon CloudTrail logs and calculates number of API calls that Amazon Macie has monitored as a part of the security analysis of your infrastructure. Total user sessions is a count of user sessions which Amazon Macie has processed. And total users shows the number of users that have been identified by CloudTrail data. The bottom of the dashboard is used to present a number of different views in graphs, charts and statistics of monitored data. These being S3 objects for the selected time range. This displays the S3 objects within a time range at a minimum risk level. S3 objects, this metric shows your monitored S3 objects grouped together by Amazon Macie themes. S3 objects by PII. This shows PII data grouped by priority and type. S3 objects by ACL. This groups S3 objects by their ACL URIs, display names, and permission levels. High-risk CloudTrail events and associated users. This metric relates to the top 20 high-risk CloudTrail events detected in the last 60 days. High-risk CloudTrail errors and associated users. This displays the errors resulting from API actions detected in the CloudTrail logs. Activity location, this represents the global map showing the locations of activity of actions that Amazon Macie is monitoring and analyzing. CloudTrail events, this identifies all CloudTrail events monitored by Macie. Activity ISPs, this records the ISPs that have been used by users. And finally, CloudTrail user identity types. This groups users detected by their identity type, such as an IAM user.
Once I had reviewed the Macie dashboard, I looked at the users section. In this lecture I covered the following points. Users are grouped by platinum, gold, silver and bronze. Which represents their perceived level of risk based on their history of API calls. Platinum poses the highest risk, and bronze the lowest risk. Additional data can be generated by selecting the user, which will present you with a condensed version of the dashboard displaying metrics only relating to that particular user. User identity types are defined from CloudTrail logs that it monitors and analyzes via the user identity element. User identity types include, root, which means the request was initiated by AWS root account. IAM user, the request was made my an IAM user. Assumed role, defines that the request was made by credentials that were temporarily assumed by the assumed role API. Federated user, the request again was made with temporary credentials, but using the STS GetFederation Token API. AWS account, the request was made by a different AWS account. And AWS service, the request was made by an AWS service.
Following this lecture, I then focused on the research feature of Amazon Macie. Here I covered the following points. The research function allows you to create your own queries against all of the data that Amazon Macie has collected and monitored via AWS CloudTrail and Amazon S3. It enables you to perform deep dive analysis of your data that relates to your specific requirements within your business using the query parser. You can filter the results based on CloudTrail data, S3 bucket properties, and S3 objects. You can also filter on the number of results found, along with a date range filter. Research is integrated with all elements of Macie, for example the dashboard and alerts. An example of an entry on the query parser is as follows. Which will only display the results with a get bucket policy API call is used by IAM users only. It's possible to save your favorite queries within a favorites list, and you can also create your queries and have them saved as a custom alert.
Following this lecture, I then moved my focus on to explain how Amazon Macie classifies your data to assign its risk value. In this lecture, I explained that every data object within the Amazon S3 bucket automatically receives a perceived level of risk based on a classification process. There are four classification categories. Content type allows Macie to detect the type of file that is being stored on S3. For example, a binary file, a document, or source code object. File extensions. This looks at the file extension of the object to ascertain its risk value. Themes. This assesses the object based on a series of key words that are detected within the actual object itself. And Regex, regular expression. This classifies content based on content within the object using a text string for describing a specific search pattern. Each S3 object has a risk value for each category. The object's final risk value is given by the highest value received between the four categories. Amazon Macie also performs automatic PII classification using a list of predefined metrics. Amazon Macie uses AI and machine learning to assess and review historical CloudTrail data access patterns using CloudTrail events and CloudTrail errors. CloudTrail events provide a list of CloudTrail events along with the associated risk value of the API. CloudTrail errors looks at the different errors that are generated and recorded within CloudTrail.
The final lecture was a demonstration where I showed you how to use a single AWS master account to gather data for multiple AWS accounts.
That now brings me to the end of this lecture, and to the end of this course. You should now be able to effectively use Amazon Macie to help you protect your data, and to meet and maintain governance and compliance regulations within your environment.
If you have any feedback on this course, positive or negative, please do contact us at email@example.com. You feedback is greatly appreciated. Thank you for your time, and good luck with your continued learning of cloud computing.
Stuart has been working within the IT industry for two decades covering a huge range of topic areas and technologies, from data center and network infrastructure design, to cloud architecture and implementation.
To date, Stuart has created 150+ courses relating to Cloud reaching over 180,000 students, mostly within the AWS category and with a heavy focus on security and compliance.
Stuart is a member of the AWS Community Builders Program for his contributions towards AWS.
He is AWS certified and accredited in addition to being a published author covering topics across the AWS landscape.
In January 2016 Stuart was awarded ‘Expert of the Year Award 2015’ from Experts Exchange for his knowledge share within cloud services to the community.
Stuart enjoys writing about cloud technologies and you will find many of his articles within our blog pages.