With so many different storage solutions available in AWS, at times it’s difficult to determine the best way to architect your storage needs in the most cost-optimized way. This course has been designed to help you understand the different costs associated with each of the AWS services currently listed under the ‘Storage’ category, and when you might select one option within a service over another one.
This course explores the pricing metrics of a variety of AWS services including Amazon S3, Glacier storage classes, Elastic File System, FSx, Storage Gateway, and AWS Backup. For any feedback, queries, or suggestions, please feel free to contact us at support@cloudacademy.com.
Learning Objectives
- Understand the various pricing metrics implemented across different storage services in AWS
- Gain a full insight into your expected costs without any hidden surprises appearing on your bill
Intended Audience
This course is suitable for:
- Storage engineers responsible for maintaining and storing data within the enterprise
- AWS Architects who are designing new solutions requiring data storage capabilities
- Individuals looking to begin their certification journey with either the AWS Cloud Practitioner or one of the three Associate-level certifications
Prerequisites
This is an intermediate-level course to AWS storage services and therefore, to get the most out of it, you should have some basic knowledge of the services discussed.
Hello and welcome to this lecture covering the elastic file system. EFS is a fully managed, highly available and durable service that allows you to create shared file systems that can easily scale to petabytes in size with low latency access. EFS has been designed to maintain a high level of throughput in addition to low latency access response, and these performance factors make EFS a desirable storage solution for a wide variety of workloads, and use cases and can meet the demands of tens, hundreds or even thousands of EC2 instances concurrently.
Being a managed service, there is no need for you to provision any file servers to manage the storage elements or provide any maintenance of those servers. This makes it a very simple option to provide file-level storage within your environment. It uses standard operating system APIs, so any application that is designed to work with standard operating system APIs will work with EFS.
It supports both NFS versions 4.1 and 4.0 and uses standard file system semantics such as strong consistency and file locking. It's replicated across availability zones in a single region making EFS a highly reliable storage service.
As the file system can be accessed by multiple instances, it makes it a very good storage option for applications that scale across multiple instances allowing for parallel access of data. The EFS file system is also regional, and so any application deployments that span across multiple availability zones can all access the same file systems providing a level of high availability of your application storage layer.
Much like Amazon S3 and Glacier, there are different storage classes within EFS, not as many, however, they can help you save a lot of money if used correctly. There are currently 2 different storage classes, these being: EFS Standard - The default storage used when using EFS And EFS Infrequent Access (EFS-IA).
As you can expect, the EFS-IA storage class is designed for files that are accessed less frequently than those recommended for the standard storage class. As a result, a considerable cost saving can be achieved of up to 92%.
The result of the cheaper storage means that there is an increased first-byte latency impact when both reading and writing data in this class when compared to that of Standard. The costs are also managed slightly differently. When using IA, you are charged for the amount of storage space used, which is cheaper than that compared to Standard. However, with EFS-IA, you are also charged for each read and write you make to the storage class. This helps to ensure that you only use this storage class for data that is not accessed very frequently, for example, data that might be required for auditing purposes or historical analysis.
With the Standard storage class, you are only charged on the amount of storage space used per month. Both storage classes are available in all regions where EFS is supported. And importantly, they both provide the same level of availability and durability.
As you can see from this table taken from the London region, EFS-IA is considerably cheaper, and also shows the additional charges for requests.
EFS also offers lifecycle management and when enabled, EFS will automatically move data between the Standard storage class and the EFS-IA storage class. This process occurs when a file has not been read or written to for a set period of days, which is configurable, and your options for this period range include 14, 30, 60, or 90 days.
Depending on your selection, EFS will move the data to the IA storage class to save on cost once that period has been met. However, as soon as that same file is accessed again, the timer is reset, and it is moved back to the Standard storage class. Again, if it has not been accessed for a further period, it will then be moved back to IA. Every time a file is accessed, its lifecycle management timer is reset. The only exceptions to data not being moved to the IA storage class is for any files that are below 128K in size and any metadata of your files, which will all remain in the Standard storage class.
Throughput modes In addition to being charged for your storage class, EFS also offers 2 different rates of throughput, which also come at a different cost.
These two different throughput modes that are offered are: Bursting Throughput and Provisioned Throughput.
Data throughput patterns on file systems generally go through periods of relatively low activity with occasional spikes in burst usage, and EFS provisions throughput capacity to help manage this random activity of high peaks.
With the Bursting Throughput mode, which is the default mode, the amount of throughput scales as your file system grows. So the more you store, the more throughput is available to you. Using the bursting throughput mode does not incur any additional charges and you have a baseline rate of 50 KB/s per GB of throughput that comes included with the price you pay for your EFS standard storage.
Provisioned Throughput allows you to burst above your allocated allowance, which is based upon your file system size. So if your file system was relatively small but the use case for your file system required a high throughput rate, then the default bursting throughput options may not be able to process your request quickly enough. In this instance, you would need to use provisioned throughput. However, this option does incur additional charges where you will need to pay for any bursting above the default capacity allowed from the standard bursting throughput.
In a way, Amazon EFS offers simpler cost options over Amazon S3, largely due to the fact that there are fewer storage classes and no charges for data retrieval. In the interest of cost-optimization, it's generally recommended that you implement the lifecycle policies to help you reduce your file system costs by moving data between standard and the infrequent access storage classes as the majority of files are accessed rarely with only a small percentage of files being accessed regularly. From a data transfer perspective, it is recommended that you use AWS DataSync which is a managed service and will help you to manage the transfer of data into and out of EFS and is charged at a flat per-GB rate. For more information on the Amazon Elastic File Systems, you can take our existing course here.
Lectures
Stuart has been working within the IT industry for two decades covering a huge range of topic areas and technologies, from data center and network infrastructure design, to cloud architecture and implementation.
To date, Stuart has created 150+ courses relating to Cloud reaching over 180,000 students, mostly within the AWS category and with a heavy focus on security and compliance.
Stuart is a member of the AWS Community Builders Program for his contributions towards AWS.
He is AWS certified and accredited in addition to being a published author covering topics across the AWS landscape.
In January 2016 Stuart was awarded ‘Expert of the Year Award 2015’ from Experts Exchange for his knowledge share within cloud services to the community.
Stuart enjoys writing about cloud technologies and you will find many of his articles within our blog pages.