Data Transfers with AWS DataSync
This course explores the AWS DataSync service which focuses on the transfer of data between on-premises into AWS storage services, in addition to transferring data between AWS storage services.
- Define what AWS DataSync is and what it's used for
- Understand AWS DataSync use cases
- Define the architecture and process of using AWS DataSync to transfer data from on-premises to AWS
- Define the architecture and process of using AWS DataSync to transfer data between different AWS storage services
This course has been designed to assist those who are responsible for managing and maintaining data and storage solutions, also it would be advantageous to anyone who is looking to take an AWS associate level exam.
To get the most out of this course, it would be beneficial to have a basic awareness of the different available AWS Storage Services, including Amazon S3, EFS, and FSx for Windows Server.
In this lecture I am going to answer the question, what is AWS DataSync? If you’ve been working with different AWS storage services for any length of time then you may have already come across this service. AWS DataSync is a service that allows you to easily and securely transfer data from your on-premise data center to AWS storage services. It can also be used to manage data transfer between 2 different AWS storage services too, so it’s a great service to help you migrate, manage, replace and move data between different storage locations.
At the time of writing this course, AWS DataSync supports the ability to work with data stored on Network File Systems shares, Server Message Block shares, and any self-managed object storage, in addition to the following AWS services:
- Amazon S3
- Amazon Elastic File System
- Amazon FSx for Windows File Server
- AWS Snowcone
When performing data transfer operations, whether this be from on-premises or between AWS storage services, DataSync support AWS VPC Endpoints and so its able to utilise the high bandwidth, low latency AWS network to it’s advantage, this helps to both simplify the management of the request and automate your data transfer across secure infrastructure. For more information on AWS Endpoints, please see our AWS Networking lecture found here: https://cloudacademy.com/course/aws-networking-features-essential-for-a-solutions-architect/vpc-endpoints/
With data transfer speed a being a key factor for a data transfer services, AWS Data Sync comes with its own purpose-built data transfer network protocol in addition to a parallel and multithreaded architecture to rapidly perform data transfer, this means that each DataSync task has the potential of utilizing 10 Gbps over a network link between your own on-prem data center and your AWS environment.
Obviously when working with data, especially when moving it between 2 points, security is a key concern. As a result AWS DataSync provides 2 levels that provide end-to-end security, these being encryption, in addition to data validation.
From an encryption perspective, encryption in transit is implemented by encrypting the data using the Transport Layer Security (TLS) protocol. When data reaches an AWS service, it also supports encryption at rest mechanisms that EFS and FSx for Windows service offers, in addition to the default encryption at rest option for Amazon S3.
The 2nd point, Data Validation, ensures that your data arrives at its destination in one piece, exactly as it was when it left the source ensuring that it wasn’t compromised or damaged in any way during its transit. This additional check helps to validate the consistency of your data that was written to the AWS storage service, and that its a perfect match from when it left its source location.
From a cost perspective, AWS DataSync usesa flat pricing strategy based on a per-gigabyte of data transferred, this makes it easy to predict avoiding any unexpected costs.
Stuart has been working within the IT industry for two decades covering a huge range of topic areas and technologies, from data center and network infrastructure design, to cloud architecture and implementation.
To date, Stuart has created 90+ courses relating to Cloud reaching over 140,000 students, mostly within the AWS category and with a heavy focus on security and compliance.
Stuart is a member of the AWS Community Builders Program for his contributions towards AWS.
He is AWS certified and accredited in addition to being a published author covering topics across the AWS landscape.
In January 2016 Stuart was awarded ‘Expert of the Year Award 2015’ from Experts Exchange for his knowledge share within cloud services to the community.
Stuart enjoys writing about cloud technologies and you will find many of his articles within our blog pages.