The AWS Solutions Architect Associate Level Certification
AWS Elastic Compute Cloud
AWS Simple Storage Service
AWS Identity and Access Management
This lecture on S3 advanced concepts builds on the previous to detail how S3 stores, syncs, and shares your data over time to improve data integrity of your data and minimize costs.
First, we will discuss how versioning of your buckets differs from suspending them, and how to verify you bucket status.
We will enter a bucket and upload a file to it, which essentially creates a new version. You will see how this new version is visible with the old, and learn how to avoid creating multiple objects, which will be difficult to managed and expensive.
Next, we will discuss the lifecycle rules of S3, and how they can keep your bucket managed easily.
We will walk through creating rules to apply to objects in your bucket; explaining how to configure them properly to archive unnecessary objects; and how Glacier’s storage power can be harnessed to lower your storage costs.
You will create notifications that are triggered by S3 events; learn about the many object classes; and determine which notification works best for you: SNS topic, SQSQ, or Lambda.
Finally, you will create a subscription for the topic for specific users to receive the notifications. You will decide how the notifications will be sent: HTTP, email, SMS, or SQS addresses. We will wrap it up by creating a topic, sending it off, and reviewing the process.
In the last video we showed how to use S3 to store, sync and share your data. Now we'll explore managing your S3 objects over the long term in ways that increase data integrity and keep costs down.
Earlier, we synced data between a local computer and an S3 bucket using the --delete argument. This meant that once a file was deleted in the local archive, it would be removed from the S3 sync bucket the next time sync was run. It would also completely overwrite older versions of the same file. This might not fit your needs in every case. AWS allows versioning to protect data against accidental deletion or overwriting. You activate versioning at the bucket level. In the all buckets page, select the bucket you'd like to work with and then, if it isn't already, click on the properties tab on the right, expand versioning, and after reading AWS's warning, always a good idea by the way, click on enable versioning. The enable versioning button has been replaced by a suspend versioning button. Suspending versioning will not effect existing objects, but will prevent future duplication. Let's try it out. We'll enter our bucket and upload a file from the local computer.
Now I've added to the local file offline and saved it to create effectively a new version. Now let's upload the updated file, which has the exact same file name as before. Notice the two versions tab at the top, but click on the show tab and both versions of our file are now visible, and are available for any purpose.
However, once you start adding thousands or even millions of objects to a bucket, it doesn't take too much imagination to see how things might get messy, not to mention extremely expensive. That's where life cycle rules can play an important role. Let's expand life cycle in our properties window and click on add rule.
We could apply this rule to all the objects in the bucket, or to all the objects in a particular folder. But for now, let's just focus on the file we've uploaded. We'll select a prefix, and then type the first few letters of the file name. Just enough to restrict this rule only to the files we're interested in.
Click on configure rule. Let's do nothing to the current version, but click on the drop down menu next to the action on previous versions. We'll select archive, and then permanently delete. Archive means transfer the file to Amazon's Glacier storage service. Glacier's advantage is lower storage cost, something that makes the slower recovery speeds a decent trade off. So let's archive our older versions 30 days after they become older versions.
That is to say, 30 days after they're overwritten or expired, and delete them after 90 days. We'll click review, and if we like, give this rule a name. And once we're satisfied with it, click create an activate rule. Buckets and their contents can have multiple life cycle rules to create a highly customized storage environment. This kind of cycling is very common in the handling of large bodies of data like log files, or sensitive customer data that can be subject to regulatory controls. Because S3 buckets don't exist in a vacuum, but are usually only one part of a larger project, they can be easily integrated into your workflows both human and programmed. Therefore, S3 events can be configured to trigger notifications.
Expand events in the properties frame of your bucket window, and then type a name for your new event. Click once in the events box to display a menu of choices.
We'll choose our RRS object lost. An event in the reduced redundancy storage, RRS object class by the way. Notifications can be sent to an SNS topic, SQSQ, or Lamda function. We'll select SNS topic. We'll just take a quick detour to explain how to create a new SNS topic. From the main AWS dashboard, click on SNS, and then on create a new topic. Enter a descriptive name for your topic and a display name.
They can be the same if you prefer. In order for this topic to be useful, we'll have to create a subscription. That is, you'll have to subscribe some recipients to the topic who will actually read about it and hopefully respond. You can have the topic notifications sent to HTTP, email, SMS, or SQS addresses. We'll add a fictional email address which in any case won't actually work until it's owner confirms. We now have an active topic. Now we can select our topic. Click inside the SNS topic box, and you'll see all the topics associated with this account, including the one we just created. Select it and click save.
David taught high school for twenty years, worked as a Linux system administrator for five years, and has been writing since he could hold a crayon between his fingers. His childhood bedroom wall has since been repainted.
Having worked directly with all kinds of technology, David derives great pleasure from completing projects that draw on as many tools from his toolkit as possible.
Besides being a Linux system administrator with a strong focus on virtualization and security tools, David writes technical documentation and user guides, and creates technology training videos.
His favorite technology tool is the one that should be just about ready for release tomorrow. Or Thursday.