Data Transfers with AWS DataSync
Amazon S3
Running Operations with the Snow Family
Amazon Kinesis
Amazon CloudFront Design Patterns
Inter-Regional and Intra-Regional Communication Patterns
Understanding Direct Connect
The course is part of this learning path
This course covers the core learning objective to meet the requirements of the 'Designing Network & Data Transfer solutions in AWS - Level 2' skill
Learning Objectives:
- Understand the most appropriate AWS connectivity options to meet performance demands
- Understand the appropriate features and services to enhance and optimize connectivity to AWS public services such as Amazon S3 or Amazon DynamoDB.
- Understand the appropriate AWS data transfer service for migration and/or ingestion
- Apply an edge caching strategy to provide performance benefits for AWS solutions
Hello and welcome to this lecture covering how you can speed up your long-distance S3 data transfers using Transfer Acceleration.
When transferring data into or out of Amazon S3 from and to your remote client, or to another AWS region, transfer acceleration can dramatically speed up the process by utilizing another AWS service, Amazon CloudFront.
Amazon CloudFront is a content delivery network service (CDN), which essentially provides a means of distributing traffic worldwide via edge locations. AWS edge locations are sites deployed in major cities and highly populated areas across the globe. More information on Amazon CloudFront can be found in our existing course here.
When transferring data to S3 from your client with transfer acceleration enabled at the bucket level, the request will go via one of the CloudFront Edge Locations, from here the transfer request will then be routed through a high speed optimized AWS network path to Amazon S3.
When using transfer acceleration you should be aware that there is a cost. Whereas normal data transfer into amazon S3 is free from the internet, with transfer acceleration, this is a cost associated per GB depending on which edge location is used. Also, there is an increased cost for any data transferred OUT of S3, either to the internet or to another Region, again due to the edge location acceleration involved.
To enable transfer acceleration is very simple to do. Select your bucket within the S3 console, select ‘Properties’ and then select the Transfer Acceleration tile.
You can then either ‘Enable’ it or ‘Suspend’ as required. You will also notice that you will be given an Endpoint, in this case, the endpoint for the bucket is as shown.
As a result, to enable transfer acceleration your bucket name must be DNS compliant and not contain any periods at all. Also, to make use of the transfer acceleration feature itself, any requests, such as GET or PUT to the bucket, must use this new transfer acceleration endpoint.
One final point to make with transfer acceleration is that there are a couple of S3 operations that it does not support, these being: GET Service (list buckets), PUT Bucket (create bucket), DELETE Bucket, and Cross region copies using PUT Object - Copy.
Stuart has been working within the IT industry for two decades covering a huge range of topic areas and technologies, from data center and network infrastructure design, to cloud architecture and implementation.
To date, Stuart has created 150+ courses relating to Cloud reaching over 180,000 students, mostly within the AWS category and with a heavy focus on security and compliance.
Stuart is a member of the AWS Community Builders Program for his contributions towards AWS.
He is AWS certified and accredited in addition to being a published author covering topics across the AWS landscape.
In January 2016 Stuart was awarded ‘Expert of the Year Award 2015’ from Experts Exchange for his knowledge share within cloud services to the community.
Stuart enjoys writing about cloud technologies and you will find many of his articles within our blog pages.