******************

UPDATE: 

Please note that the File Sync feature has now evolved into a new AWS Service known as AWS DataSync.  More information on this service can be found here

******************

Resources Referenced

• Supported Regions for EFS
• Amazon EFS Pricing page

Lecture Transcript

Hello and welcome to this lecture covering the Elastic File System, known as EFS. EFS provides a file level storage service, whereas the likes of EBS provided block level storage and Amazon S3, object level storage. EFS is a fully managed, highly available and durable service, that allows you to create shared file systems that can easily scale with your demands, and meet requests by tens, hundreds, or even thousands of ec2 instances concurrently. Being a managed service, there is no need for you to provision file service to manage the storage elements, or provide any maintenance of those servers. This makes it a very simple option to provide file level storage within your environment. From a capacity perspective, it's very similar to S3, in that it's seemingly limitless. As you store more and more files using EFS, your storage simply elastically grows with the request. There is no need to provision a set size of data storage, like you do with EBS for example, where you need to specify the size of the volume that you require. As the file system can be accessed by multiple instances, it makes it a very good storage option for applications that scan across multiple instances, allowing for parallel access of data. The EFS file system is also regional, and so any application deployments that span across multiple availability zones can all access the same file systems, providing a level of high availability of your application's storage layer. At the time of writing this course, EFS is not currently available within all regions. For a list of supported regions, please visit the following link. EFS has been designed to maintain a high level of throughput, in addition to low latency access response. These performance factors make EFS a desirable storage solution for a wide variety of workload and use cases. 

To create your EFS file system, it's a very simple and quick process, which can be managed from within the AWS management console. From within the EFS dashboard, you select to create a new file system, and then you are required to enter some configurational information. You must select which VPC that this file system will exist in, and once selected, EFS will automatically create mount targets for you, across the availability zones where you have EC2 instances. These mount targets allow you to connect to the EFS file system, form your EC2 instances, using a configured mount target IP address. When mounting the EFS file system, be aware that it is only compatible with NFS version 4, and 4.1. With this in mind, EFS does not currently support the Windows operating system. You must insure that your Linus EC2 instance has the NFS client installed for the mounting process, and the NFS client version 4.1 is recommended for this procedure. For each mount point, you are able to select which subnet the mount point exists in, as well as defining your security group to control access from what instance level. 

The next step of creating your file system involves defining your tags, performance mode, and encryption settings. The two different performance mode of operations are general purpose and Max I/O. For most use cases and requirements, you will likely be using general purpose. It has the lowest latency out of the two different modes, and will work with many different application workloads. There is a limitation of this mode, allowing only up to 7,000 file system operations per second to your EFS file system. If, however, you have huge scale architectures, or your EFS file system is likely to be used by many thousands of EC2 instances concurrently, and will exceed 7,000 operations per second, then you will need to consider Max I/O. This also offers a virtually unlimited amount of throughput and IOPS, in addition to additional latency to each I/O. 

The best way to understand which performance option you need is to run tests alongside your application. If your application sits comfortably within the upper limit of the 7,000 operations per second, then general purpose will be best suited, with the added plus point of the lower latency. However, if your testing confirms 7,000 operations per second may be tight, then select Max I/O. When using the general purpose mode of operations, EFS provides a CloudWatch metric, PercentIOLimit, which allows you to view your operations per second as a percentage of the top 7,000 limit. This allows you to make the decision to migrate and move to the Max I/O file system, should your operations be reaching the limit. 

You also have the opportunity to implement encryption of your EFS file system, using a simple checkbox, and selecting your desired CMK. Much like EBS, EFS uses the services offered by the key management service, to provide encryption of crucial storage. However at this stage, encryption is only offered at rest, and not in transit. The final stage requires you to review and create your EFS file system, based on the configuration that you have specified. 

Once your file system is created, you are presented with your mount target points, allowing you to connect your EC2 instances as required. This shows the mount targets generated from the configuration given in these screenshots. In addition to having the ability of being able to mount the new EFS file system to your EC2 instances in your VPC, you can always use these mount points on your on-premise service, as long as you connect via direct connect, or 3rd party VPN. 

If you have existing data, either on your own service in your own on-premises data center, or data that already exists in AWS, such as on EC2 instances and you want to move that data into EFS, then you can use the file sync feature. File sync can be configured from within the EFS dashboard of the management console, and it allows you to securely manage the transfer of files between an existing storage location, and your EFS file system via a file sync agent. If you require the need to sync source files from your on-premises environment, then you can download the file sync agent as a VMware ESXi host. If you are syncing source files from within AWS, then it will provide a community-based AMI, to be used with an EC2 instance. This agent is then configured with your source destination amount target of your EFS file system details, and logically sits in between them. From within the EFS dashboard, you can then start the file sync, and monitor it's progress with Amazon CloudWatch. 

The pricing structure is very simple for EFS. There are no charges for data transfer, or any charges for requests, like we have in Amazon S3 in Glacier. You are, however, charged for the amount of data you consume in per gigabyte-months. This allows for the calculation of space used, as it fluctuates over the month. For example, let's say from within the Ireland region, you used 80 gig of storage for 10 days in March, and 175 gig of storage for the rest of March, totaling 21 days. During March, you would have the following usage in gigabyte hours. This is then converted to gigabyte month, to calculate the final figure. For the latest information on EFS pricing, please visit the Amazon EFS pricing page, as shown on the screen. 

As I have already explained, EFS provides file level storage capabilities. As a result, this doesn't make it an ideal solution for data archiving. Instead, we would use Amazon Glacier. Also, relational database data is better suited to EBS, and finally, EFS is not recommended for temporary storage. This should be fulfilled by EC2 instance store volumes instead.

Lectures:

• AWS Storage Services Overview
• AWS Storage Services
• Amazon Simple Storage Service
• Amazon Glacier
• EC2 Instance Storage
• EBS Storage
• Amazon Elastic File System
• Amazon Cloudfront
• AWS Storage Gateway
• AWS Snowball
• Summary