In this lesson of the Cloud Practitioner course, we introduce you to the various migration and transfer services currently available in AWS that are relevant to the CLF-C02 exam.
- Identify and describe the various migration services available in AWS
- Describe AWS services used for transferring data from on-premises environments
This lesson is designed for anyone who is new to cloud computing, so no prior experience with AWS is necessary. While it may be helpful to have a basic understanding of AWS and its services, as well as some exposure to AWS Cloud design, implementation, and operations, this is not required as all of the concepts we will introduce in this lesson will be explained and reinforced from the ground up.
In this lecture, I will be looking at different scenarios to help you decide which snow device to use and when.
So we have the following snow devices AWS Snowcone, AWS Snowball, and AWS Snowmobile.
The AWS Snowcone is the smallest of the snow family, this has been designed to be lightweight, easily portable, allowing you to easily use the device pretty much anywhere and under any conditions due to the ruggedness of the casing, and the added advantage of being able to run on battery should a persistent mains connection not be available. It can easily fit into a standard backpack, and AWS have even demonstrated that the snowcone can be attached to a drone emphasising its portability and versatility. Packed with 8TB of storage and an EC2 instance, this device is perfect for taking your computing needs way beyond the cloud and your Data Centre allowing you to capture, process, and analyze data, perhaps via IoT sensors, which can then be shipped back to AWS for data transfer, or you could even use AWS DataSync to transfer the data on-line over your traditional network connectivity.
For those unaware of AWS DataSync, it’s a service that allows you to easily and securely transfer data from your Snowcone or your on-premise data center, to AWS storage services. It can also be used to manage data transfer between 2 different AWS storage services too, so it’s a great service to help you migrate, manage, replace and move data between different storage locations.
This is essentially the elder sibling of the Snowcone, it’s bigger in size and it contains a greater amount of storage and compute power. This brings a new set of use cases for this device, it’s primarily used for large scale data transfer operations, up to 80TB at a time, both in and out of AWS. The devices themselves can be rack mounted in your data centre, and if need be clustered in groups of 5-10 devices. Unlike Snowcones, they can’t be powered by battery expansion packs, and they are not as portable, for example, you can’t stick a snowball in a backpack and walk up a mountain, or strap it to a drone!
The Storage optimized snowball is targeted for data migrations and transfers with its storage being compatible with both S3 object storage and EBS volumes. The Compute optimized snowball is a great option if you need to handle compute intensive edge computing workloads in disconnected environments. From a storage perspective, it also comes with 42 TB of usable HDD capacity which comes compatible with EBS volumes and S3 object storage. The Compute Optimized with GPU option is used to accelerate AI, HPC, and graphics, which is great when working with video analysis and graphic intensive use cases.
Both the Snowcone and Snowball can be used for many of the same use cases which I will reference in just a moment, so if that’s the case, when would you use the snowcone over the snowball and vise versa?
You would use the snowcone if you:
- Needed a portable device that you could easily carry to difficult to reach locations and situations
- Only needed a maximum of 8TB storage
- If you needed the ability to perform on-line data transfer using AWS DataSync, preventing you the need to send the Snowcone back to AWS for an off-line data transfer
- If you didn’t have a consistent power support and you needed the support of a battery pack
Alternatively, you would use the Snowball device if you:
- Didn’t need to provide mobility to the snow device and it could remain in one location for a set period of time
- Needed to transfer data of up to 80TB
- Needed the ability to run enhanced graphics processing by using the Compute Optimized with GPU option
- Had a requirement to transfer data using S3 API’s
- Required the use of usable SSD Storage
- Needed to optimized network ports that could reach speeds of up to 100Gbit, as Snowcones only have network port speeds of 10Gbit
- Needed to cluster your snowballs. Clustering allows you to order between 5-10 snowball devices, acting as a single pool of resources. This allows you to gain a larger storage capacity, and also enhance the level of durability of the data should a snowball fail. Clustering is only an option if you are looking to simply perform local compute and storage workloads without transferring any data back to AWS.
- Needed to rack mount your devices providing the opportunity to implement temporary installations of both compute and storage
- Required the snow device to be HIPAA compliant
Ok, so that should provide a better understanding of how the Snowcone and Snowball differ. Let me now run through a couple of scenarios of when you might use these devices in the real world.
Both the snowcone and Snowballs are perfectly suited to provide a level of portable edge computing allowing you to collect data from wireless sensors or networked resources, for example in locations such as industrial warehouses or manufacturing plants, where you might need to collect environmental metric data. By collecting and gathering data it can then be transferred to AWS offline, or if using the snowcone it can be transferred on-line using AWS DataSync, which can then be analyzed at scale using other AWS services.
With storage capabilities of up to 80TB of usable HDD storage from a single snowball, it easily allows you to provide a means of securely storing and transferring a large amount of data into AWS, and you can run multiple snowballs in parallel allowing you to transfer petabytes of data if required. Being of rugged design and portable, the devices can be used in remote locations, such as mining and oil sectors, or even in the travel industry, fitted to trucks, trains, and boats, providing a mechanism of easily collecting data and then transferring it back to AWS.
Another common use case is from within the media and entertainment industry, the Snowcone and snowball can be used as a way to aggregate data from multiple sources before shipping it back to AWS for transfer into Amazon S3. You might get video and audio data from multiple feeds, especially if you are working in the film industry, this data can then be aggregated to your snowball device and shipped back to AWS for further processing and editing from your wider production team.
So we have covered Snowball and Snowcone, but let’s now turn the attention to the AWS Snowmobile, what is the primary use case for this? Well, it’s quite simple, the AWS Snowmobile is used to transfer MASSIVE amounts of data from a single location, up to 100PB per snowmobile which arrives on a truck as a ruggedized shipping container. When you are talking about data transfer of this scale you are normally looking at migrating entire data centers to a new location, or migrating entire storage libraries or repositories, and so AWS snowmobile is a great solution to help you with this when you need it done quickly, securely and cost-efficiently. You can run multiple snowmobiles in parallel which will allow you to transfer Exabytes of data! Generally, you would use AWS snowmobiles if you needed to transfer more than 10petabytes of data, anything less than this then you might want to consider using multiple snowball devices.
Stuart has been working within the IT industry for two decades covering a huge range of topic areas and technologies, from data center and network infrastructure design, to cloud architecture and implementation.
To date, Stuart has created 150+ courses relating to Cloud reaching over 180,000 students, mostly within the AWS category and with a heavy focus on security and compliance.
Stuart is a member of the AWS Community Builders Program for his contributions towards AWS.
He is AWS certified and accredited in addition to being a published author covering topics across the AWS landscape.
In January 2016 Stuart was awarded ‘Expert of the Year Award 2015’ from Experts Exchange for his knowledge share within cloud services to the community.
Stuart enjoys writing about cloud technologies and you will find many of his articles within our blog pages.