Amazon S3 Lifecycle Configurations
Introduction to Amazon EFS
EFS in Practice
Amazon Elastic Block Store (EBS)
Running Operations with the Snow Family
Data Transfers with AWS DataSync
The course is part of this learning path
This section of the Solution Architect Associate learning path introduces you to the core storage concepts and services relevant to the SAA-C03 exam. We start with an introduction to the AWS storage services, understand the options available and learn how to select and apply AWS storage services to meet specific requirements.
Want more? Try a lab playground or do a Lab Challenge!
- Obtain an in-depth understanding of Amazon S3 - Simple Storage Service
- Get both a theoretical and practical understanding of EFS
- Learn how to create an EFS file system, manage EFS security, and import data in EFS
- Learn about EC2 storage and Elastic Block Store
- Learn about the services available in AWS to optimize your storage
- Learn how to use AWS DataSync to move data between storage systems and AWS storage services
Amazon FSx is another storage service that focuses on file systems, much like EFS. However, FSx comes in 2 forms, firstly Amazon FSx for Windows File Server, and also Amazon FSx for Lustre. Each FSx option has been designed for very different needs and requirements. We can summarize the differences as shown in the following table.
Amazon FSx for Windows File Server Provides a fully managed native Microsoft Windows file system on AWS. You can easily move and migrate your windows-based workloads requiring file storage. The solution is built upon Windows Server, it operates as shared file storage. It has full support for: SMB protocol, Windows NTFS, Active Directory (AD) integration, and Distributed File System (DFS). And it uses SSD storage for enhanced performance and throughput providing sub-millisecond latencies.
Amazon FSx for Lustre is a fully managed file system designed for compute-intensive workloads, for example, Machine Learning and high-performance computing. It has the ability to process massive data sets. Performance can run up to hundreds of GB per second of throughput, millions of IOPS, and sub-millisecond latencies. It has integration with Amazon S3 and supports and supports cloud-bursting workloads from on-premises over Direct Connect and VPN connections.
As you can see, they are both designed to solve very different solutions depending on your workloads.
The pricing for each of these options operates in a different way, firstly, let’s take a look at the FSx for Windows, which has 3 price points: Capacity, Throughput, and Backups.
Again, much like EFS, there are no setup fees for the use of this service, however, you do pay for the amount of storage capacity that you consume. This is priced on the average storage provisioned per month and uses the metric of gigabyte-months and offers varied pricing between a single or multi-AZ deployment.
In addition to the actual storage that you use there is also a cost of for the amount of throughput that you configure for your file systems, this metric is based upon MBps-months. Again, cost variations exist between single and multi-AZ deployment. One point to bear in mind is that any data transfer costs when using multi-AZ is included in the pricing you see for the multi-AZ deployment.
Because Amazon FSx performs incremental backups (either manual or automatic) of your file systems, it optimizes your storage costs as only the changes since the last backup are saved. Much like your storage capacity, backup storage is also charged based on the average metric of gigabyte-months, meaning the average amount of capacity you have used in the month.
This table shows the cost for both single and multi-az deployments for each of the price points I just discussed. As you can see, there is a significant increase in cost should you need to deploy your FSx file system across multiple AZs.
One benefit that you currently get with Amazon FSx for Windows File Server over Amazon FSx for Lustre is that of data deduplication. This can be activated for your file system and can save you up to 80% is storage costs depending on what data you are storing. Data deduplication simply means FSx will automatically store duplicate files or portions of files a single time, this helps you save on your storage capacity costs and remains invisible to the user that the data has been deduplicated.
As you can see in the table, savings range from 30% - 80% depending on what type of content is being saved within your file system. Data deduplication doesn’t add any additional costs, so it’s a great way to help optimize and reduce your storage costs.
The costs associated with Amazon FSx for Lustre differs to that of Windows File Server and offers a simpler cost model.
Again, there are no set up fees and you only pay for the storage capacity that you use, based on the GB-month metric. For data transfer between availability zones you will pay the standard data transfer costs for the associated region.
Depending on your workload and use case for your data you will use either Windows Server or Lustre. The Windows Server option does offer more flexibility from a pricing point of view with the added advantage of data deduplication to help save up to 80% of storage costs. Although it does also have a cost associated with throughput capacity, which Amazon Lustre does not.
For more information on Amazon FSx for Lustre, please see our existing blog post here.
Stuart has been working within the IT industry for two decades covering a huge range of topic areas and technologies, from data center and network infrastructure design, to cloud architecture and implementation.
To date, Stuart has created 150+ courses relating to Cloud reaching over 180,000 students, mostly within the AWS category and with a heavy focus on security and compliance.
Stuart is a member of the AWS Community Builders Program for his contributions towards AWS.
He is AWS certified and accredited in addition to being a published author covering topics across the AWS landscape.
In January 2016 Stuart was awarded ‘Expert of the Year Award 2015’ from Experts Exchange for his knowledge share within cloud services to the community.
Stuart enjoys writing about cloud technologies and you will find many of his articles within our blog pages.