This course covers the core learning objective to meet the requirements of the 'Designing storage solutions in AWS - Level 3' skill
Learning Objectives:
- Evaluate the different Amazon S2 encryption meachanisms available for both client and serfver cryptographic operations
- Create a file storage strategy for complex organizations
- Analyze the differenr AWS storage services available to implement a hybid storage solution based upon different data set sizes, business requirements, and cost optimization
Let’s say you own a company - called Company A. Company A is not fully in the cloud. Instead, they work mainly with on-premises infrastructure that looks like this: there are servers that have access to critical data locally stored in traditional storage arrays.
Company A has quickly realized that there are issues with this picture - especially when it comes to storing their data. For example, what if the company wants to scale out? They’ll find that there are upper-limits to how far they can scale storage - not only the financial limitation of how much they can afford, but also the resource limitation - company A might not have the people resources or the physical space to store, maintain, and administer this infrastructure.
Or they may be looking to reduce their on-premises infrastructure and eventually go all-in on cloud. In that case, procuring tons of expensive hardware doesn’t make sense long-term.
And last, they might also struggle to provide a central location for data access, especially if they have an on-premises presence in multiple locations around their country. They might determine that doing this with traditional storage arrays is less convenient, more expensive, and more difficult to do.
For all of these reasons and more, company A wants a hybrid storage solution. To do this, they use AWS Storage Gateway to enable hybrid storage between their on-premises and cloud infrastructure.
So now, their infrastructure looks like this: they still have their on-premises application servers and storage, but they also now have a connection to the AWS Cloud through the Storage Gateway.
This gateway can be deployed in three main ways. You can choose to deploy it as:
-
as a virtual machine in your on-premises environment, such as VMware ESXi, Microsoft Hyper-V, or Linux KVM.
-
The gateway could also be deployed with the AWS Storage Gateway Hardware Appliance, which is a standalone pre-configured piece of hardware that you can deploy into your on-premises environment.
-
Or three, instead of deploying it on-premises, you can deploy it in your AWS environment as an EC2 instance or in VMware Cloud on AWS.
When you install your gateway, you’ll need to allocate at least 150 MB of local disk space to the deployment. This is used for the local cache. The gateway uses this local cache for two main purposes:
-
It uses it as a staging area for data that the gateway will upload to AWS.
-
And it can be used as a true cache, to save data for low-latency access. That way when the application requests data, the gateway will check the cache storage for that data before it downloads it from AWS, ideally saving time to serve the request.
This gateway then connects to the Storage Gateway service over a secure, encrypted connection. This service is responsible for the communication to other AWS services and manages the transfer of data to S3, FSx, EBS, Amazon Glacier, and more. Where the data is transferred and how you interact with that data is dependent on what type of storage gateway you choose. There are four types of storage gateway: S3 File Gateway, FSx File Gateway, Tape Gateway, and Volume Gateway.
S3 File Gateway enables you to store objects in Amazon S3 using the NFS or SMB protocol. FSx File Gateway enables you to store objects in Amazon FSx for Windows File Server using the SMB protocol.
The Tape Gateway replaces physical tape libraries with iSCSI-VTL(or virtual tape libraries). Each virtual tape is stored in Amazon S3 and can be exported to lower-cost storage tiers like Amazon S3 Glacier Flexible Retrieval and Amazon S3 Glacier Deep Archive for long-term storage.
And last, the final option is the volume gateway, which enables you to store data in block storage using the iSCSI protocol.
So what do you pay for the storage gateway service? Well there are three main pricing components:
-
Storage pricing
-
Request pricing
-
And Data transfer pricing.
Storage and request pricing differs based on the type of Gateway you use. For example, if you use an S3 File Gateway, the storage pricing is the same as S3 storage pricing and if you use the FSx File Gateway, the storage pricing is the same as FSx for Windows storage pricing. You are also billed for requests at the S3 and FSx rates.
For data transfer, you aren’t charged for data transfer into AWS, but you are charged for data transfer out of the storage gateway service to your on-premises gateway. This is charged based on how many TB a month you transfer out to your on-premises gateway. As always, for the most up to date information on AWS pricing, make sure you check out the AWS pricing documentation.
In summary, your on-premises applications communicate to your deployment of Storage Gateway. You can choose from four different types of storage gateway, which determines the protocols you can use to communicate to this gateway. The gateway then communicates to the Storage Gateway service that connects to storage services like S3, EFS, and EBS. You are mainly charged for storage, request, and data transfer costs.
All right, that’s all for this one, see you next time.
Stuart has been working within the IT industry for two decades covering a huge range of topic areas and technologies, from data center and network infrastructure design, to cloud architecture and implementation.
To date, Stuart has created 150+ courses relating to Cloud reaching over 180,000 students, mostly within the AWS category and with a heavy focus on security and compliance.
Stuart is a member of the AWS Community Builders Program for his contributions towards AWS.
He is AWS certified and accredited in addition to being a published author covering topics across the AWS landscape.
In January 2016 Stuart was awarded ‘Expert of the Year Award 2015’ from Experts Exchange for his knowledge share within cloud services to the community.
Stuart enjoys writing about cloud technologies and you will find many of his articles within our blog pages.