Performance Factors Across AWS Storage Services
In this course, I will discuss different performance factors across AWS storage services.
- Identify the proper AWS storage services to use within your architectures as well as various configuration options within each service that can impact their performance, the differences between block, object, and file storage, as well as the benefits they offer and proper use cases for each
- The different services AWS offers for each of these storage types
- How you can configure the storage types to optimize overall performance
- Anyone who needs to determine the right storage services and configurations to use within their AWS Cloud-based architectures to maximize performance and meet other architectural requirements
- Have a basic understanding of AWS storage services, as well as experience designing and implementing solutions in the AWS Cloud
Hello, and welcome to this lecture, where I will discuss performance factors with different AWS file storage services. And for our file storage services in AWS, we’re going to be looking at both Amazon EFS and FSx. So as the name suggests, file storage is used to maintain a hierarchy of files that live inside of folders. And these folders can contain both files and potentially additional folders as well, leading to a nested file path structure. And file storage is probably the most commonly used type of storage for our network-shared file systems using network-attached storage, or NAS devices.
File storage offers many advantages over other types of storage, including the ability for files to be accessed concurrently from multiple different clients as if they were on a local hard drive. And it’s important to remember that file storage is just an abstraction on top of block storage, using a file system like Network File System, known as NFS, or Server Message Block, known as SMB.
So again in AWS, we have the Elastic File System or EFS, which is a fully managed file storage service for use with Linux-based EC2 instances and on-premises servers, as well as Lambda functions and other AWS container services such as ECS, EKS, and Fargate. It’s massively scalable and capable of being accessed by thousands of instances at a time. To learn more about EFS before we discuss its performance, I encourage you to check out this course.
And we also have Amazon FSx, which comes in several different offerings supporting a range of different workloads including FSx for Lustre, Windows File Server, OpenZFS, and NetApp ONTAP. If you want to learn more about FSx before we dive into its performance, check out this course.
So now let’s talk about performance factors when it comes to both EFS and FSx. In terms of EFS, when we first create our file system we have the ability to configure a storage class, a performance mode, and a throughput mode. Now your choice of storage class, which can be either One Zone or Standard, will not have any impact on performance, as this only affects whether your file system is highly available. But it should come as no surprise that your choice of performance mode will definitely have an impact on, well, performance. So your choices for performance modes are General Purpose and Max I/O. Now if your file system is using the One Zone storage class, it must use the General Purpose performance mode.
And this mode will give you very low latency, on the order of microseconds for read operations and milliseconds for write operations, but has a maximum performance of only 35,000 IOPS for read operations and 7,000 IOPS for write operations. On the other hand, Max I/O will increase your latency somewhat, but it will also drastically increase your maximum IOPS to over 500,000 IOPS for read operations and over 100,000 IOPS for write operations. Now neither of these performance modes change the maximum throughput of your file system, which will range from 3-5 gigabytes per second per file system read, and 1-3 gigabytes per second per file system write.
Now in terms of throughput, we can choose either Bursting or Provisioned throughput modes. And Bursting is the default mode, which allows you to accumulate burst credits inside of burst buckets during periods of low throughput. These burst credits can then be used to maintain performance during periods of higher throughput demand. And Bursting is generally a good choice for most situations where your throughput needs are going to vary over time. But if you have constant and predictable throughput needs, you should opt to use Provisioned Throughput mode instead. With Provisioned Throughput, you’ll be charged based on the amount of throughput that exceeds the base throughput rate of the file system as if you were using Bursting Throughput mode. For more information on EFS performance factors, please check out the AWS documentation here.
Now when it comes to Amazon FSx, all of the different file system offerings will support sub-millisecond latency and throughput of at least 2-3 gigabytes per second. In the case of FSx for Lustre, maximum throughput is actually much higher at 1000 gigabytes per second, and Lustre file systems can also scale up to petabytes in size. You can see more of the performance specifications for different flavors of FSx in this table:
Now you’ll configure an initial throughput capacity for your FSx file system at the time you create it. And your choice of capacity will have an impact on the overall performance of your file system. For instance, with FSx for Windows File Server, you can choose throughput capacities that range from 32 all the way up to 2,048 megabytes per second. And based on your choice, a few other performance options will scale along with it, including:
Network throughput capacity, or the speed at which the file server can serve out files to clients over the network;
File server memory, which can be used to support background processes like copying and deduplicating file data;
File server disk throughput; and
But keep in mind that you can always adjust this throughput capacity up or down over time to meet your requirements as well. So hopefully that gives you a better sense of the performance factors for your EFS and FSx file systems.
Danny has over 20 years of IT experience as a software developer, cloud engineer, and technical trainer. After attending a conference on cloud computing in 2009, he knew he wanted to build his career around what was still a very new, emerging technology at the time — and share this transformational knowledge with others. He has spoken to IT professional audiences at local, regional, and national user groups and conferences. He has delivered in-person classroom and virtual training, interactive webinars, and authored video training courses covering many different technologies, including Amazon Web Services. He currently has six active AWS certifications, including certifications at the Professional and Specialty level.