Amazon S3 Lifecycle Configurations
Introduction to Amazon EFS
EFS in Practice
Amazon Elastic Block Store (EBS)
Running Operations with the Snow Family
Data Transfers with AWS DataSync
The course is part of this learning path
This section of the Solution Architect Associate learning path introduces you to the core storage concepts and services relevant to the SAA-C03 exam. We start with an introduction to the AWS storage services, understand the options available and learn how to select and apply AWS storage services to meet specific requirements.
Want more? Try a lab playground or do a Lab Challenge!
- Obtain an in-depth understanding of Amazon S3 - Simple Storage Service
- Get both a theoretical and practical understanding of EFS
- Learn how to create an EFS file system, manage EFS security, and import data in EFS
- Learn about EC2 storage and Elastic Block Store
- Learn about the services available in AWS to optimize your storage
- Learn how to use AWS DataSync to move data between storage systems and AWS storage services
Hello and welcome to this lecture. AWS Storage Gateway allows you to provide a gateway between your own data center's storage systems such as your SAN, NAS or DAS and Amazon S3 and Glacier.
The Storage Gateway itself can either be installed as software or physical hardware appliance that can be stored within your own data center which allows integration between your on-premise storage and that of AWS. This connectivity can allow you to scale your storage requirements both securely and cost-efficiently.
Storage Gateway offers different configurations and options allowing you to use the service to fit your needs. It offers file, volume and tape gateway configurations which you can use to help with your DR and data backup solutions, and each of these come with different price points.
File Gateways. File gateways allow you to securely store your files as objects within S3. This connectivity acts as a type of file share allowing you to mount or map drives to an S3 bucket as if the share was held locally on your own corporate network. In addition to this, a local on-premise cache is also provisioned for accessing your most recently accessed files to optimize latency which also helps to reduce egress traffic costs.
As this service integrates largely with Amazon S3, much of the pricing is based upon S3 price points. As we can see in the table below for the London region, there are 2 metrics of pricing associated with File Gateways, Storage pricing and Request pricing. As the storage used for this type of Gateway solely relies on S3, the actual storage costs are as per the Amazon S3 storage class price at the time of use which I reviewed in a previous lecture.
When we look at the request pricing there is a small cost per GB associated to data written to S3 by the Storage Gateway, up to a maximum of $125.00 per gateway per month. Also, the first 100 GB is free. All other request types again are as per Amazon S3’s pricing.
Volume Gateways. Volume Gateways can be figured in one of two different ways: Stored volume gateways and cached volume gateways.
Stored volume gateways are often used as a way to backup your local storage volumes to Amazon S3 as EBS snapshots whilst ensuring your entire data library also remains locally on-premise for very low latency data access. Volumes created and configured within the storage gateway are backed by Amazon S3 and are mounted as iSCSI devices that your applications can then communicate with.
With Cached volume gateways, the primary data storage is actually on Amazon S3 rather than your own local storage solution. However cache volume gateways utilize your local data storage as a buffer and the cache for recently accessed data to help maintain low latency, hence the name, Cache Volumes.
Although volume gateways still utilize Amazon S3, they do not, however, follow the S3 pricing mechanism like File Gateways do. Stored volume gateways create EBS snapshots, which are then stored on S3, however, they are billed as Amazon EBS snapshots. The cached volumes, however, are charged on a per GB-month metric of data stored.
Any requests are priced similarly to File gateways in that they are billed on a per GB basis of data written by the Gateway, up to a maximum of $125.00 per gateway per month. Also, the first 100 GB is free, in addition to any deletes to EBS volumes or snapshots also remain free.
Tape Gateways. The final option with AWS Storage Gateway is a tape gateway known as Gateway VTL. Virtual Tape Library. This allows you again to back up your data to S3 from your own corporate data center in addition to being able to leverage the storage classes within Glacier for data archiving for a far lower cost than S3. Virtual Tape Library is essentially a cloud-based tape backup solution replacing physical components with virtual ones.
From a cost point of view, you should be aware of Virtual Tapes. These are a virtual equivalent to a physical backup tape cartridge and any data stored on the virtual tapes are backed by AWS S3/Glacier and appear in the virtual tape library. A Virtual Tape Library, VTL, as you may have guessed are virtual equivalents to a tape library that contain your virtual tapes.
Much like both the File and Volume gateways, the pricing is split across storage and request pricing.
There are 3 different options for Storage pricing of Tape Gateways and these are S3, S3 Glacier, and S3 Glacier Deep Archive. All of which are charged at per GB-month of data stored. Generally, if you are using Tape Gateways you are looking to take advantage of the very low price points of Glacier and Deep Archive which offer significant savings as you can see in the table.
Request pricing also offers a range of different cost metrics depending on the type of action and storage class used.
Much like File and Volume gateways, there is a small cost per GB associated to data written to S3 by the Storage Gateway, up to a maximum of $125.00 per gateway per month. Also, the first 100 GB is free.
For any virtual tape retrieval requests that are being stored on S3 Glacier classes, you will also pay a per GB cost, with Deep Archive providing a more expensive retrieval rate.
Any request that results in your moving your virtual tapes between your S3 Glacier storage class and S3 Glacier Deep Archive you will pay a fee per GB of data moved.
If you do select the Glacier storage classes and you delete your virtual tapes, within a set time period (90 days for S3 Glacier and 180 days for Deep Archive), then you will be charged a prorated charge per month per GB.
AWS Storage Gateway uses a variety of storage options, from Amazon S3, EBS Snapshots, Amazon S3 Glacier and S3 Glacier Deep Archive, and the cost of each is dependent on the type of gateway required which will, of course, be dictated by your use case.
File gateways pricing is very simple and essentially follows the pricing metrics of Amazon S3, apart from a per-GB request for data writes. Volume gateways again offer a simple pricing structure but uses the per-GB metric for volumes in addition to any snapshots of the volumes priced at EBS snapshot costings.
Tape gateways offer additional complexity due to the range of storage classes that it can use. So when using Tape gateways, understand the retrieval times for your data based on the S3 Glacier and Deep Archive classes as you might be able to save a considerable amount when using these classes depending on the criticality of the data you might need to retrieve.
For a deeper give on all things AWS Storage Gateway, please see our existing course here which will dive deeper into this service and how each gateway type operates and works with regards to connectivity.
Stuart has been working within the IT industry for two decades covering a huge range of topic areas and technologies, from data center and network infrastructure design, to cloud architecture and implementation.
To date, Stuart has created 150+ courses relating to Cloud reaching over 180,000 students, mostly within the AWS category and with a heavy focus on security and compliance.
Stuart is a member of the AWS Community Builders Program for his contributions towards AWS.
He is AWS certified and accredited in addition to being a published author covering topics across the AWS landscape.
In January 2016 Stuart was awarded ‘Expert of the Year Award 2015’ from Experts Exchange for his knowledge share within cloud services to the community.
Stuart enjoys writing about cloud technologies and you will find many of his articles within our blog pages.