Amazon S3 Introduction
S3 Management Features
S3 Security Features
Amazon S3 Encryption
Amazon Elastic Block Store (EBS)
Introduction to Amazon EFS
EFS in Practice
The course is part of this learning path
This section of the SysOps Administrator - Associate learning path introduces you to the core storage concepts and services relevant to the SOA-C02 exam. We start with an introduction to the AWS storage services, understand the options available, and learn how to select and apply AWS storage services to meet specific requirements.
- Obtain an in-depth understanding of Amazon S3 management and security features
- Get both a theoretical and practical understanding of EFS
- Learn how to create an EFS file system, manage EFS security, and import data in EFS
- Learn about EC2 storage and Elastic Block Store
Hello, and welcome to this lecture that is going to look at the different methods of using policies and permissions to gain access to S3 resources, which include both S3 buckets and objects within those buckets. We can control access to S3 resources via policies using both identity-based policies and resource-based policies.
Identity-based policies are attached to the IAM identity requiring access, and then using IAM permission policies, either in-line or managed, they can then be associated to the user, a group that the user belongs to, or via a role that the user has permission to assume. Identity-based policies define the resource in the policy.
For example, the bucket name, as you can see in this example policy shown here. So this identity-based policy can be associated with a user and will permit them to use all S3 actions within the bucket, s3deepdive.
You can of course use conditions within your policies to manage and refine access to your resources even further. For example, in this policy, the identity would be able to perform the actions CreateBucket, DeleteBucket, and PutObject on the s3deepdive bucket on the condition that their source IP address was within the range of the CIDR block 10.1.0.0/16.
For a full list of actions, resource types, and condition keys to use within policies, please see the AWS reference here.
Now, resource-based policies differ in the fact that the policy is associated with a resource rather than the identity. So from an Amazon S3 perspective, resource-based policies come in the form of Access Control Lists and bucket policies. Because of this, you need to define within the policy who will be allowed or denied access. This is managed differently depending on if you're using bucket policies or Access Control Lists.
Let me first focus on bucket policies. A bucket policy is written in JSON and is directly attached to your bucket as another means of granting and restricting access control. And by default, when you create a bucket, no bucket policy exists.
To add a modified bucket policy, you must select the bucket, go to permissions, and then select the Edit button in the bucket policy section. From here, you have one of three different options. You can either freely write your own bucket policy in the JSON window, view some policy examples, or use the AWS policy generator to help you create your bucket policy.
When working with bucket policies, you must specify a principle element, which defines the principal who is associated with the action and effect defined in the statement. Let's look an example bucket policy to make it clearer. This bucket policy allows the actions, s3 DeleteObject and s3 PutObject, for the s3deepdive bucket for the principal Stuart.
You'll notice that with the identity-based policy shown previously, I didn't need to add the Principle element. It's important to note that the permissions defined within the statements of a bucket policy apply to the bucket and the objects that reside within the bucket.
So now, we are aware that you can use both IAM policies to control access to S3 data and bucket policies, you might be wondering when you should use one over the other. So let me take a look at that in more detail. You might want to use IAM if you want to centrally manage your access control methods all in the one service, that being IAM.
Also, if you have multiple permissions to apply to a large number of buckets, the management of doing so might be easier to do from within IAM access across one or two policies rather than creating separate bucket policies that you would have to attach to each of your buckets. Also, IAM has the added advantage of being able to control access for more than one service at a time within its policies, whereas bucket policies only control access to the S3 bucket and its objects.
On the flip side, you might want to use bucket policies if you want to maintain your security policies within S3 alone. You can also grant cross-account access using bucket policies without having to create and assume roles that are created within IAM.
For example, all you'd need to do is to create a bucket policy as shown and attach it to your bucket, for example, s3deepdive, replacing the text in red with the external account. In the external account, users then simply have to be delegated access to the same bucket to be able to access the bucket.
Also, another reason you might want to use bucket policies is because IAM policies can be a maximum of two kilobytes in size for users, five kilobytes for groups, and 10 kilobytes for roles. However, bucket policies can reach a size of 20 kilobytes.
Now, you can of course use both IAM policies and bucket policies to control access. They are not mutually exclusive. They can both be used together to control access. I should come to how multiple access controls are evaluated and their logic after we look at S3 Access Control Lists, ACLs, which as I mentioned previously are another resource-based access control method.
S3 Access Control Lists, or ACLs, allow you to control access to buckets in addition to specific objects within a bucket by groupings and AWS accounts. One advantage of being able to use ACLs is that you can set different permissions per object.
ACLs do not follow the same JSON format as policies defined by IAM and bucket policies. Instead, they are far less granular, and different permissions can be applied depending if you are applying an ACL at the bucket level or the object level.
Due to the basic structure of an ACL, it is not possible to implicitly deny access using ACLs. Neither are you able to implement conditional elements, like we saw earlier when I mentioned identity-based access. An example of default ACL for a bucket looks as shown.
As you can see here, you can specify different permissions for different predefined S3 groups defined by the Grantee column. Permissions are based on the grantee of which there are four:
- The bucket owner. This'll be your own AWS account and will have full control over all objects and the bucket itself.
- Everyone (public access). Permissions set against this grantee would mean anyone can access using the permissions applied, providing the object had been made public. These requests can be signed, authenticated, or unsigned, unauthenticated.
- Authenticated users. This option will allow IAM users from any AWS account to access the object via signed request, authenticated.
- S3 log delivery group. This is a group used to deliver log files when server access logs has been configured, and the bucket is used to store and write log files to.
By editing the ACL at the bucket level, you'll be presented with the following. This shows you the different levels of permissions that can be applied to the bucket. It will show you the permissions given to the grantee groups for the bucket, which are either List or Write. You can also see what permissions have been given to enable the grantee access to either read or write against the bucket ACL.
For clarification about what these permissions actually grant, they are as follows. List, which allows the grantee to list objects in the bucket. Write, which allows a grantee to create, overwrite, and delete objects in the bucket. Bucket ACL Read, which allows the grantee to read the ACL of the bucket. And Bucket ACL Write, which allows the grantee to write the ACL for the bucket.
You may have noticed that you can also add access for another account as a grantee. To configure this access, you'll be required to enter the canonical ID of the AWS account that you would like to have access. So we have seen what ACL looks like at the bucket level.
Let's now take a quick look at what it looks like at the object level. Here's an ACL of an object within a bucket. As you can see, again, by default, the resource owner has full control over the object. Also, you will notice that you can add access for another account.
Again, you'll need to add the appropriate canonical AWS account ID and the relevant permissions. You can also specify public access if you have public access enabled, which I will explain more on later. In this example, we can see that the Everyone group has read access to the object.
To modify these permissions, I could simply click on the radio button next to the group and modify the settings as shown. Note, there is no write permission when working with object ACLs, unlike bucket ACLs.
Again, let's take a closer look at what actual permissions are being applied depending on the option chosen for the object. Read object allows the grantee to read the object data and its metadata. Read object permissions allows the grantee to read the object ACL. Write object permissions allows the grantee to write the ACL for the object.
So we have now looked at a number of different methods to implement access controls with various permissions to your S3 buckets and objects. We looked at IAM policies, S3 bucket policies, and finally S3 Access Control Lists. But what would happen if you used all of these access control methods? How would the policy evaluation logic operate? Which would take precedence?
Let's say, for example, someone tries to access an object in a bucket that has S3 ACLs attached in addition to bucket permissions and also have their own IAM permissions. How is this access governed if there are conflicting permissions to the object in the bucket that they are trying to access? All of these policies will be viewed together to determine the resulting access and will handle any permission conflict in accordance with the principle of least privileged.
Essentially, by default, AWS states that access is denied to an object, even without an explicit Deny within any policy. To gain access, there has to be an Allow within a policy that the principal is associated to or defined by within a bucket policy or ACL. If there was no Deny defined, but there is an Allow within a policy, then access will be authorized. However, if there is a single Deny associated with the principal to a specific object, then even if an Allow does exist, this explicit denial will always take precedence overruling the Allow and access will not be authorized.
So going back to our example, we have a user called Stuart who has the following identity-based policy associated. The bucket s3deepdive also has the attached bucket policy. The ACL permissions on the s3deepdive bucket give the bucket owner full control to the bucket and its objects. The user Stuart is a part of the same AWS account as the bucket owner.
So what are the resulting permissions? The IAM permissions grant Stuart all actions to the s3deepdive bucket, the bucket policy denies Stuart access to delete buckets and objects within the s3deepdive bucket, and the ACL allows access for Stuart, as he is a part of the account relating to the bucket owner.
So as we know, a Deny always takes precedence over an Allow, meaning Stuart will have access to the s3deepdive bucket to perform all S3 actions apart from deleting the bucket or any of its objects.
Stuart has been working within the IT industry for two decades covering a huge range of topic areas and technologies, from data center and network infrastructure design, to cloud architecture and implementation.
To date, Stuart has created 90+ courses relating to Cloud reaching over 140,000 students, mostly within the AWS category and with a heavy focus on security and compliance.
Stuart is a member of the AWS Community Builders Program for his contributions towards AWS.
He is AWS certified and accredited in addition to being a published author covering topics across the AWS landscape.
In January 2016 Stuart was awarded ‘Expert of the Year Award 2015’ from Experts Exchange for his knowledge share within cloud services to the community.
Stuart enjoys writing about cloud technologies and you will find many of his articles within our blog pages.