Storage on AWS
Elastic Block Store
Simple Storage Service
Advanced services: Glacier and Storage Gateway
Storage is a central part of any computing infrastructure. Amazon provides many services on the cloud to replace traditional, on-premises storage systems, ranging from short-term storage for running instances who are doing computation on smal batches of data, up to long-term archives saved in redundant disks or even tapes.
In this course, the Computer Engineer and Cloud Expert Mohammad Ali Tabibi will give you an overview of the AWS storage services like EBS, S3, Glacier and Storage Gateway, to better understand what they are for, how they are built, and how they can be best used.
Who should take this course
Being a beginner course, no prerequisites are needed to understand the concepts of this course. Nevertheless, having some knowledge of what AWS is, and having some experience with the Linux Command Line Interface, might be helpful to follow along the course.
If you want to test your knowledge on the basic topics covered in this course, we strongly suggest to take our AWS questions. Also, if you want to learn more about the other AWS services, please consider checking out our other AWS courses.
Now let's see how to attach a five gigabyte general purpose Solid-State Drive to an EC2 instance with Ubuntu Server 14.04 long term support HVM AMI. We start with opening the Amazon EC2 console. As you can see in the EC2 Dashboard, one running instance and one available volume exists.
Now, choose the available volume from the volume section, click on action and choose attach volume. In the instance box you can select the instance you want to attach the volume to. Now verify that the suggested device name is suitable or enter a new device name in the device box. Note that depending on the block device driver of your instances kernel, the device may be attached with a different name than what you specify. For example, if you specify a device name of /dev/sdh your device might be renamed to something like /dev/xvdh or /dev/hdh by the kernel. In some cases even the trailing letter may also change.
Finally, click attach to attach the volume to the instance. Remember that the volume and instance must be the same availability zone. After a few seconds, you can see the state of the volume change to in-use and it's exposed as a block device. You can format the volume with the file system you like and then mount it. After you can make the Amazon EBS volume available for use and you can access it in the same way as you access any other volume. You can also take snapshots of your EBS volume for backup purposes or to use it as a base line when you create another volume.
After attaching a volume to an instance, we need to make it available. The AMI of this instance is Ubuntu Server 14.04 LTS HVM, so to connect to this instance you can use an SSH client in Linux or Mac OS or software like PuTTY in Windows which is what we'll use now. First of all let's connect to our machine.
Once we are in, we can use the lsblk command to view our available disk devices and their mount points, if applicable, to help us determine the correct device name to use. Please note that the output of lsblk removes the /dev prefix from full device paths. In this example as you can see, /dev/xvda1 is mounted as the root device and /dev/xvdf is attached but it has not yet been mounted. Now we need to understand if we have to create a file system on the volume. New volumes are raw block devices and we need to create a file system on them before we can mount and use them. We can use the file command to get this information. If the output of the previous command shows simply data for the device, then there is no file system on the device and we need to create one. If you need to create a file system on the volume, you can use the mkfs commands to create ext4 file system. Depending on the requirements of your application or the limitations of your operating system, you may opt through a different file system type.
Finally, let's create a mount point for the volume and then mount the volume at that location. This can be done with mkdir and mount commands respectively. Now the volume is available and ready to use.
As we already said, Amazon EBS provides the ability to save point-in-time snapshots of your volumes to Amazon S3.
EBS snapshots are stored incrementally. Only the blocks that have changed after your last snapshot are saved and you're billed only for the changed blocks. If you have a device with 100 gigabytes of data but only five gigabytes has changed after your last snapshot, a subsequent snapshot only consumes an additional five gigabytes and you're only billed for the additional five. Even though both the earlier and later snapshots appear complete. Snapshots can be used to instantiate multiple new volumes, expand the sizes of the volume or move volumes across availability zones.
When a new volume is created, you may choose to create it based on the existing EBS snapshot. In that scenario, the new volume begins as an exact replica of the snapshot. To create a snapshot, open the Amazon EC2 console, click snapshots in the navigation pane.
The console displays a list of current snapshots, click create snapshot. As you see, the create snapshot dialogue box appears. Now select the volume to create a snapshot for then click create, write a name and the description for the snapshot.
Finally, just click create. Amazon EC2 starts creating the new snapshot. With an EBS-backed instance you can back up the root device volume of the instance by creating a snapshot. When you have a snapshot of the root device volume of an instance you can terminate that instance and then later launch a new one from the snapshot itself. This can be useful if you don't have the original AMI you launched, but you need to be able to launch an instance using the same image. Let's see how to do that.
Open the Amazon EC2 console and then in the navigation pane under elastic block store click snapshots. Now select the snapshot you just created and select create image from the actions list. In the create image from EBS snapshot dialogue box, fill the fields to create your AMI as you can see and then click create.
Now you have your newly created AMI in the AMI section and you can use it to launch a new instance based on that. You just need to choose it from the AMIs list, click launch and follow the provided instructions.
EBS snapshots have a number of key features: immediate access to Amazon EBS volume data. The created volume from a snapshot is accessible immediately because Amazon EBS snapshots implement lazy loading. So you can begin using them right away. Resizing Amazon EBS volumes, it's possible to specify a larger size for a new volume when you want to create it based on a snapshot. Sharing Amazon EBS snapshots, your Amazon EBS snapshots can be shared with others in the AWS community. Copying Amazon EBS snapshots across AWS region. For geographical expansion, data center migration and disaster recovery it's possible to copy snapshots across AWS regions. For use with EBS, Amazon EC2 has EBS-optimized instances.
They allow you to fully use the IOPS provision done in EBS volume. EBS-optimized instances deliver dedicated throughput between Amazon EC2 and Amazon EBS with options between 500 and 2,000 megabits per second depending on the instance type used. The dedicated throughput minimizes contention between Amazon EBS input/output and other traffic from your EC2 instance providing the best performance for your EBS volumes.
Computer Engineer and Cloud Expert