Amazon S3 Security: master S3 bucket polices and ACLs

Learn about Bucket Policies and ways of  implementing Access Control Lists (ACLs) to restrict/open your Amazon S3 buckets and objects to the Public and other AWS users.

Follow along and learn ways of ensuring the public only access for your S3 Bucket Origin via a valid CloudFront request.

Welcome to part 8 of my AWS Security Series. This week I shall be looking at some of the security features around the Simple Storage Service (S3). In particular, Bucket Policies and how you can implement Access Control Lists (ACLs) to restrict or open up your S3 buckets and objects to the Public and other AWS users. I will also cover how you can ensure the public only access your S3 Bucket Origin via a valid CloudFront request, ensuring CloudFront is not bypassed resulting in unauthorised access.

S3 Security

If you are looking to implement security on S3 then you would already be familiar with what the service is and its reliability as a Storage service and the benefits it can bring. With this in mind you are probably storing a lot of data on this service and as a result you will want to ensure that it’s safe and secure! I will run through some of the security elements of S3 that you can choose to deploy, depending on your data’s sensitivity.

Bucket Policies

Bucket Policies are similar to IAM policies in that they allow access to resources via a JSON script. However, Bucket policies are applied to Buckets in S3, where as IAM policies are assigned to user/groups/roles and are used to govern access to any AWS resource through the IAM service.
When a bucket policy is applied the permissions assigned apply to all objects within the Bucket. The policy will specify which ‘principles’ (users) are allowed to access which resources. The use of Principles within a Bucket policy differs from IAM policies, Principles within IAM policies are defined by who is associated to that policy via the user and group element. As Bucket policies are assigned to Buckets, there is this need of an additional requirement of ‘Principles’.

Example Policy

Policy example
As shown above and if you read my previous article ‘Creating an AWS IAM Policy’, the syntax is very similar to IAM Policies. As mentioned previously though, there is the addition of the ‘Principal’ section. In the example above the principal is listed as user ‘CloudAcademy1’ via the users ARN which can be found in IAM.
The example policy allows ‘CloudAcademy1’ access to Delete Objects and Put Objects within the ‘cloud-academy’ Bucket.

Setting Bucket Policy Conditions

Again similarly to IAM Policies, S3 Bucket Policies allow you to set conditions with the Policy, for example allowing specific IP subnets to access the Bucket and perhaps restricting a specific IP address. The example below shows how to implement such conditions.
S3 policies
In the ‘Condition’ section above you can see that the Subnet of 192.168.1.0/24 is allowed to access the bucket, however the IP address from this range of 192.168.1.10 is not allowed via the NotIpAddress condition.

For a full list of conditions and help on creating your S3 Bucket Policies take a look at the great tool that AWS provides here: AWS Policy Generator

An explicit deny within the policy will always take precedence over an ‘allow’. Access of least privilege will always over-rule where conflicts between policies exist. This will also be the case if you have an IAM user with S3 access to a specific bucket, which also happens to have a Bucket Policy. AWS will look at both policies and apply access on a least-privilege condition if there are conflicting permissions.
For more information on the syntax of policies and how to create and write your own, please visit my previous article here that will explain to how to create these.

S3 Access Control Lists

In addition to IAM Policies and Bucket Policies, S3 also has an additional method of granting access to specific objects through the use of Access Control Lists (ACLs), allowing a more finely grained access approach than a Bucket Policy. ACLs allow you to set certain permissions on each individual object within a specific Bucket. Again, access will always be granted on a least privileged condition if conflicts exist between ACLs, Bucket Polices and IAM Policies.

ACLs can be managed and configured from within the S3 Service itself or via APIs.

To modify Bucket ACL permissions within S3 within the Console

• Open the AWS console and select the S3 Service
• Navigate to the bucket you want to modify permissions on at an ACL level
• Select the ‘Property’ tab and then ‘Permissions’
Bucket
The permissions set here act as the ACL of the Bucket. You will notice a Grantee for the Bucket, which is the resource owner and is likely to have Full Control over that object and on now Bucket creation this is typically the AWS Account owner.
Other permissions that can be set are List (Read), Upload/Create (Write), View Permissions and Edit Permissions. If all checkboxes are selected, that Grantee is considered to have Full Control of the object.

  • You can either modify the current ACL displayed for your Bucket by selecting/deselecting the tick boxes as required, or you can add addition access by selecting ‘Add more permissions’
  • This will generate an additional line for a new Grantee. The Grantee options are as follows:
    o Everyone – This will allow access to this object by anyone, and that doesn’t just mean any AWS users, but anyone with access to the Internet
    o Authenticated AWS Users – This option will only allow IAM users or other AWS Accounts to access the object via a signed request
    o Log Delivery – This allows logs to be written to the Bucket when it is being used to store server access logs
    o Me – This relates to your current IAM AWS User Account
  • Select the appropriate Grantee and apply the permissions required using the tick boxes and then click ‘Save’
  • Your Bucket ACL has now been updated

It is worth mentioning that an S3 ACL can have up to 100 Grantees.
There are slightly different permission options between a Bucket ACL and an Object ACL as shown below.

To modify Object ACL permissions within S3 within the Console

  • Open the AWS console and select the S3 Service
  • Navigate to the object you want to modify permissions on at an ACL level
  • Select the ‘Property’ tab and then ‘Permissions’

S3 service
You will notice a small difference between the permissions available at a Bucket level to the permissions available at the Object level. Here you have the option to Open/Download the object, View Permissions and Edit Permissions.

  • You can either modify the current ACL displayed for your Bucket by selecting/deselecting the tick boxes as required, or you can add addition access by selecting ‘Add more permissions’
  • The example ACL pictured above only allows the Grantee, which happens to be the AWS Account owner, access to this file. As a result if I try to view this file via a browser using the URL highlighted within the example above I get the following “Access Denied” message:
Error
  • To allow access I need to modify the ACL to allow the Grantee of ‘Everyone’ and select the Open/Download permissions

S3
This new access will allow anyone to access the file via the URL, so when I try again now I am able to download the *.rtf file successfully and access the object. Allowing this access enables you to make your Objects accessible to the public via the assigned URL.

Using S3 as an Origin for CloudFront (Content Delivery Network – CDN)

This section of my article assumes you already have knowledge of CloudFront and its features. However, I just want to cover a point on how to implement an additional security point if you are not already doing so for your objects when using an S3 Bucket as your Origin.
When you use S3 as your Origin for CloudFront everyone has Read permission for the objects in your bucket allowing anyone to access the content via the CDN. However, if anyone or an application has the unique URLs to the objects then this will bypass the features offered by CloudFront such as access times of that object and IP restrictions, this maybe a security concern that you want to alleviate.
To ensure that no-one can access your Origin Bucket unless they are going via your CDN then you can create an Origin Access Identity (OAI) and associate that with your CloudFront Distribution. You can then allow ONLY the OAI access to your Bucket using methods already discussed in this article and remove all other user access. This way, only users utilising CloudFront will be able to access the Origin Bucket and anyone with a direct URL link will be denied as only the OAI will have access.
Creating an Origin Access Identity and assigning it to your Distribution

  • Log onto your AWS Console and select the CloudFront Service
  • Click ‘Create Distribution’
  • Select either your Web or RTMP Distribution and ‘Get Started’
  • In the ‘Origin Setting’ section we are interested in the information with the red box below
Create Distribution
  • Under ‘Restrict Bucket Access’ select ‘Yes’. This will then allow you to configure the ‘Origin Access Identity’
  • If you do not already have an OAI then select ‘Create a new identity’
  • If you want the CloudFront distribution to automatically update the permissions on your Origin Doman Name (S3 Bucket) then under ‘Grant Read Permissions on Bucket’ select ‘Yes, Update Bucket Policy’ – Note, this will allow the correct permissions for your OAI to access your Origin Bucket, however it will not remove other user access that may previously exist. You will need to go into your Polices etc to ensure any access is removed for other users that you do not want to have access
  • Continue to deploy your Distribution as you normally would

What are the key elements we have learned this week?

  • There are a number of different options available to restrict access to S3 objects, including:
    o IAM Policies
    o Bucket Policies
    o Access Control Lists
  • The difference between the above access options
  • We understood that objects will take on the access of the least privileges when conflictions conditions occur between different policies
  • We looked at how you can use an Object Access Identity to force users to access your Content via CloudFront ensuring additional features are implemented

Next week I shall be covering additional elements of the Simple Storage Service such as lifecycle policies and versioning and how they can be used to minimise data loss. There is also a feature that helps with accidental deletion that will also be covered. Lastly I will cover elements of encryption with S3 from a Client and Server side perspective.

There is a recent post that that details the pros and cons of using Amazon S3 vs Amazon Glacier: which AWS storage tool should you use?

For a reference on AWS IAM Security Nitheesh Poojary has another strong article published in August. Worth a read IMHO.

Thank you for taking the time to read my article. If you have any feedback please leave a comment below.

Avatar

Written by

Stuart Scott

Stuart is the AWS content lead at Cloud Academy where he has created over 40 courses reaching tens of thousands of students. His content focuses heavily on cloud security and compliance, specifically on how to implement and configure AWS services to protect, monitor and secure customer data and their AWS environment.

Related Posts

Avatar
Michael Sheehy
— August 19, 2019

What Exactly Is a Cloud Architect and How Do You Become One?

One of the buzzwords surrounding the cloud that I'm sure you've heard is "Cloud Architect." In this article, I will outline my understanding of what a cloud architect does and I'll analyze the skills and certifications necessary to become one. I will also list some of the types of jobs ...

Read more
  • AWS
  • Cloud Computing
Avatar
Nitheesh Poojary
— August 16, 2019

Boto: Using Python to Automate AWS Services

Boto allows you to write scripts to automate things like starting AWS EC2 instances Boto is a Python package that provides programmatic connectivity to Amazon Web Services (AWS). AWS offers a range of services for dynamically scaling servers including the core compute service, Elastic...

Read more
  • Automated AWS Services
  • AWS
  • Boto
  • Python
Avatar
Andrew Larkin
— August 13, 2019

Content Roadmap: AZ-500, ITIL 4, MS-100, Google Cloud Associate Engineer, and More

Last month, Cloud Academy joined forces with QA, the UK’s largest B2B skills provider, and it put us in an excellent position to solve a massive skills gap problem. As a result of this collaboration, you will see our training library grow with additions from QA’s massive catalog of 500+...

Read more
  • AWS
  • Azure
  • content roadmap
  • Google Cloud Platform
Avatar
Adam Hawkins
— August 9, 2019

DevSecOps: How to Secure DevOps Environments

Security has been a friction point when discussing DevOps. This stems from the assumption that DevOps teams move too fast to handle security concerns. This makes sense if Information Security (InfoSec) is separate from the DevOps value stream, or if development velocity exceeds the band...

Read more
  • AWS
  • cloud security
  • DevOps
  • DevSecOps
  • Security
Avatar
Stefano Giacone
— August 8, 2019

Test Your Cloud Knowledge on AWS, Azure, or Google Cloud Platform

Cloud skills are in demand | In today's digital era, employers are constantly seeking skilled professionals with working knowledge of AWS, Azure, and Google Cloud Platform. According to the 2019 Trends in Cloud Transformation report by 451 Research: Business and IT transformations re...

Read more
  • AWS
  • Cloud skills
  • Google Cloud
  • Microsoft Azure
Avatar
Andrew Larkin
— August 7, 2019

Disadvantages of Cloud Computing

If you want to deliver digital services of any kind, you’ll need to estimate all types of resources, not the least of which are CPU, memory, storage, and network connectivity. Which resources you choose for your delivery —  cloud-based or local — is up to you. But you’ll definitely want...

Read more
  • AWS
  • Azure
  • Cloud Computing
  • Google Cloud Platform
Joe Nemer
Joe Nemer
— August 6, 2019

Google Cloud vs AWS: A Comparison (or can they be compared?)

The "Google Cloud vs AWS" argument used to be a common discussion among our members, but is this still really a thing? You may already know that there are three major players in the public cloud platforms arena: Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP)...

Read more
  • AWS
  • Google Cloud Platform
  • Kubernetes
Avatar
Stuart Scott
— July 29, 2019

Deployment Orchestration with AWS Elastic Beanstalk

If you're responsible for the development and deployment of web applications within your AWS environment for your organization, then it's likely you've heard of AWS Elastic Beanstalk. If you are new to this service, or simply need to know a bit more about the service and the benefits th...

Read more
  • AWS
  • elastic beanstalk
Avatar
Stuart Scott
— July 26, 2019

How to Use & Install the AWS CLI

What is the AWS CLI? | The AWS Command Line Interface (CLI) is for managing your AWS services from a terminal session on your own client, allowing you to control and configure multiple AWS services and implement a level of automation. If you’ve been using AWS for some time and feel...

Read more
  • AWS
  • AWS CLI
  • Command line interface
Alisha Reyes
Alisha Reyes
— July 22, 2019

Cloud Academy’s Blog Digest: July 2019

July has been a very exciting month for us at Cloud Academy. On July 10, we officially joined forces with QA, the UK’s largest B2B skills provider (read the announcement). Over the coming weeks, you will see additions from QA’s massive catalog of 500+ certification courses and 1500+ ins...

Read more
  • AWS
  • Azure
  • Cloud Academy
  • Cybersecurity
  • DevOps
  • Kubernetes
Avatar
Stuart Scott
— July 18, 2019

AWS Fundamentals: Understanding Compute, Storage, Database, Networking & Security

If you are just starting out on your journey toward mastering AWS cloud computing, then your first stop should be to understand the AWS fundamentals. This will enable you to get a solid foundation to then expand your knowledge across the entire AWS service catalog.   It can be both d...

Read more
  • AWS
  • Compute
  • Database
  • fundamentals
  • networking
  • Security
  • Storage
Avatar
Adam Hawkins
— July 17, 2019

How to Become a DevOps Engineer

The DevOps Handbook introduces DevOps as a framework for improving the process for converting a business hypothesis into a technology-enabled service that delivers value to the customer. This process is called the value stream. Accelerate finds that applying DevOps principles of flow, f...

Read more
  • AWS
  • AWS Certifications
  • DevOps
  • DevOps Foundation Certification
  • Engineer
  • Kubernetes