In this course, we explain how to set up a hybrid storage solution using AWS Storage Gateway.
Learning Objectives
- AWS Storage Gateway and its architecture
- Which type of storage gateway to use for your use case
- How to create a storage gateway using the EC2 platform
Intended Audience
- Those that are looking to implement a hybrid storage solution
- Those who have an interest in AWS storage gateway that may need a bit more information on how it works
Prerequisites
- You should have a strong understanding of storage in AWS, including knowledge of Amazon S3, Amazon FSx, Amazon EBS, and Amazon Glacier
- Familiarity with Amazon EC2 will help as well
- For more information on these services, check out existing content here:
Let’s say you own a company - called Company A. Company A is not fully in the cloud. Instead, they work mainly with on-premises infrastructure that looks like this: there are servers that have access to critical data locally stored in traditional storage arrays.
Company A has quickly realized that there are issues with this picture - especially when it comes to storing their data. For example, what if the company wants to scale out? They’ll find that there are upper-limits to how far they can scale storage - not only the financial limitation of how much they can afford, but also the resource limitation - company A might not have the people resources or the physical space to store, maintain, and administer this infrastructure.
Or they may be looking to reduce their on-premises infrastructure and eventually go all-in on cloud. In that case, procuring tons of expensive hardware doesn’t make sense long-term.
And last, they might also struggle to provide a central location for data access, especially if they have an on-premises presence in multiple locations around their country. They might determine that doing this with traditional storage arrays is less convenient, more expensive, and more difficult to do.
For all of these reasons and more, company A wants a hybrid storage solution. To do this, they use AWS Storage Gateway to enable hybrid storage between their on-premises and cloud infrastructure.
So now, their infrastructure looks like this: they still have their on-premises application servers and storage, but they also now have a connection to the AWS Cloud through the Storage Gateway.
This gateway can be deployed in three main ways. You can choose to deploy it as:
-
as a virtual machine in your on-premises environment, such as VMware ESXi, Microsoft Hyper-V, or Linux KVM.
-
The gateway could also be deployed with the AWS Storage Gateway Hardware Appliance, which is a standalone pre-configured piece of hardware that you can deploy into your on-premises environment.
-
Or three, instead of deploying it on-premises, you can deploy it in your AWS environment as an EC2 instance or in VMware Cloud on AWS.
When you install your gateway, you’ll need to allocate at least 150 MB of local disk space to the deployment. This is used for the local cache. The gateway uses this local cache for two main purposes:
-
It uses it as a staging area for data that the gateway will upload to AWS.
-
And it can be used as a true cache, to save data for low-latency access. That way when the application requests data, the gateway will check the cache storage for that data before it downloads it from AWS, ideally saving time to serve the request.
This gateway then connects to the Storage Gateway service over a secure, encrypted connection. This service is responsible for the communication to other AWS services and manages the transfer of data to S3, FSx, EBS, Amazon Glacier, and more. Where the data is transferred and how you interact with that data is dependent on what type of storage gateway you choose. There are four types of storage gateway: S3 File Gateway, FSx File Gateway, Tape Gateway, and Volume Gateway.
S3 File Gateway enables you to store objects in Amazon S3 using the NFS or SMB protocol. FSx File Gateway enables you to store objects in Amazon FSx for Windows File Server using the SMB protocol.
The Tape Gateway replaces physical tape libraries with iSCSI-VTL(or virtual tape libraries). Each virtual tape is stored in Amazon S3 and can be exported to lower-cost storage tiers like Amazon S3 Glacier Flexible Retrieval and Amazon S3 Glacier Deep Archive for long-term storage.
And last, the final option is the volume gateway, which enables you to store data in block storage using the iSCSI protocol.
So what do you pay for the storage gateway service? Well there are three main pricing components:
-
Storage pricing
-
Request pricing
-
And Data transfer pricing.
Storage and request pricing differs based on the type of Gateway you use. For example, if you use an S3 File Gateway, the storage pricing is the same as S3 storage pricing and if you use the FSx File Gateway, the storage pricing is the same as FSx for Windows storage pricing. You are also billed for requests at the S3 and FSx rates.
For data transfer, you aren’t charged for data transfer into AWS, but you are charged for data transfer out of the storage gateway service to your on-premises gateway. This is charged based on how many TB a month you transfer out to your on-premises gateway. As always, for the most up to date information on AWS pricing, make sure you check out the AWS pricing documentation.
In summary, your on-premises applications communicate to your deployment of Storage Gateway. You can choose from four different types of storage gateway, which determines the protocols you can use to communicate to this gateway. The gateway then communicates to the Storage Gateway service that connects to storage services like S3, EFS, and EBS. You are mainly charged for storage, request, and data transfer costs.
All right, that’s all for this one, see you next time.
Alana Layton is an experienced technical trainer, technical content developer, and cloud engineer living out of Seattle, Washington. Her career has included teaching about AWS all over the world, creating AWS content that is fun, and working in consulting. She currently holds six AWS certifications. Outside of Cloud Academy, you can find her testing her knowledge in bar trivia, reading, or training for a marathon.