In-memory caching with Elasticache


Using EC2 for Databases
Start course

Databases are among the most used applications in the cloud (or anywhere else, for that matter). Managing data is exactly what computers were invented for, so it should come as no surprise that a great deal of attention is focused on the many different Database Management Systems and Data Management tools that are available.

This second part of our mini series covering AWS databases is about some of Amazon's advanced solutions. You will learn more about Redshift, the solution for massive petabyte-scale data warehouse, Elasticache, the Redis- and Memcached-based solution for in-memory cache, and SimpleDB, an easy alternative for NoSQL databases.

Who should take this course

For this beginner course, you'll require no special prerequisites. Nevertheless, some experience with Databases and at least a basic knowledge of the related jargon might be helpful. If you are completely new to the cloud, you might benefit from our introduction to cloud computing course. You might also find the AWS general introduction course interesting if you are not yet that familiar with the AWS cloud platform.

If you want to test your knowledge of the basic topics covered by this course, we strongly suggest you take our quiz questions. And of course, the first part of this course is a must if you want to learn more about the two major DB services on AWS: RDS and DynamoDB


Amazon ElastiCache is a web service to deploy, operate, and scale an in-memory cache in the Cloud. The service improves the performance of web applications by allowing you to retrieve information from fast, managed in m-memory caches instead of relying entirely on slower disk based databases. ElastiCache supports two open sourced caching engines. Memcached is a widely adopted memory object caching system. ElastiCache's protocol compliant with Memcached so popular tools that you use today with existing Memcached environments will work seamlessly with this service. Redis is a popular open source in-memory key value store that supports data structures such as sorted sets and lists. ElastiCache supports Redis Master/Slave replication which can be used to achieve cross AZ redundancy. Amazon ElastiCache automatically detects and replaces failed nodes reducing the overhead associated with self managed infrastructures and provides a resilient system that mitigates the risk of overloaded databases which slow website and application load times. Amazon ElastiCache can be used to significantly improve latency and throughput for many read-heavy application workloads such as social networking, gaming, media sharing, and Q&A portals or compute-intensive workloads such as a recommendation engine.

Caching improves application performance by storing critical pieces of data and memory for low latency access. Cached information may include the results of IO intensive database queries or the result of computationally intensive calculations.

Applications needing a data structure server will find the Redis engine most useful. The Amazon ElastiCache data model includes several important concepts. Let's look at them in detail. A cache node is the smallest building block of an ElastiCache deployment. Each node has its own memory, storage, processing resources, and runs in a dedicated instance of engine software, either Memcached or Redis.

ElastiCache provides a number of different cache node configurations for you to choose from, depends on your needs. You can use these cache nodes on an in demand basis or take advantage of reserved cache nodes at significant cost savings. A cache cluster is a collection of one or more cache nodes and each runs in its own instance of supported software. You can launch a new cache cluster with a single operation, Create Cache Cluster, and you can specify the number of nodes you want, the run time parameters, and different setting for each node. Each node in a cache cluster has the same compute storage and memory specifications and they all run in the same cache engine software. Cache parameter groups are just an easy way to manage run time sets for the support of engine software. Both of the supportive engine softwares have many parameters to control. Item sizes, cache eviction policies, and more, and a cache parameter group is just a named collection of specific parameters that you can apply to a cache cluster thereby guaranteeing that all the nodes in that cluster are configured in exactly the same way. If you're running a single node Redis cache cluster you can create a replication group for that cluster and then add one or more replicas. A replication group consists of one primary cache cluster for handling read-write traffic, plus additional cache clusters that act as read-only replicas of the first. ElastiCache ensures that the read replicas are continually synchronized with the first and in the event of a primary cache node failure, it rebuilds the primary using the data from one of the replicas. In this way, replication groups allow you to scale your read-only traffic to protect you against data loss. For enhanced security, ElastiCache nodes access restricted applications on a white listed, Amazon EC2 instance. You can control the Amazon EC2 instances, that you can add to access your cache cluster, by using cache sub-net groups and cache security groups. By default, all new ElastiCache clusters are launched in an Amazon virtual private Cloud.

You can use a cache sub-net group to grant cache cluster access from an EC2 instance running on a specific sub-net.

If you choose to run your cache cluster outside of a VPC, you can create a cache security group to authorize EC2 instances running within a specific Amazon EC2 security group. To use ElastiCache you need an AWS account. If you don't already have one you'll be prompted to create one when you sign up. After you've signed up for ElastiCache, you can create a cache cluster. In this step we use the AWS Management Console to create a cache cluster. You can use either Memcached or Redis as the cache engine software for this cluster. Redis cache clusters always have one node and Memcached cache clusters can have between one and 20 nodes. If you're new to ElastiCache, by default your cache cluster will be launched in the Amazon VPC environment. To launch a cache cluster after signing into the AWS management console, open the Amazon ElastiCache console. From the ElastiCache console dashboard, click "Get Started Now" to start the Launch Cache Cluster Wizard.

On the Cache Cluster Details page, in the Name box, type your preferred name for the cache cluster. Then, according to your needs and the previous information, choose suitable cache engine software for your cache cluster from the Engine box. As you can see, in this example we'll choose Memcached software for our engine cluster.

In the Cache Port text box you can specify a new port number for your cluster or leave the default value. For Memcached, the default value is 11211 and for Redis the default port number is 6379. In the Engine VErsion box the latest version of the software has been chosen by default so you don't need to change that. In the Number of Nodes text box enter the number of nodes you want to create. As mentioned before, for Redis the number is one and for this example we want to have two nodes so just type two.

Also, you can choose your preferred zone from the Preferred Zone box if you have a reason for that. Otherwise, just leave "No Preference". Okay, now we should choose the type of node or nodes that we want to deploy according to the memory size. As you can see, we chose cache. T1.micro which has 213 megabytes of memory for this example. As mentioned before, if you're a new ElastiCache user, a VPC is automatically created on your behave along with a cache sub-net group named "Default". In the topic for SNS Notifications box click "Disable Notifications" or select an existing SNS topic, or manually enter a topic ARN. In this example, we just disable this feature. Then you can specify if the cache cluster should receive auto minor version upgrades when they're available or not which has been selected "Yes" by default and we're just going to leave it alone.

Now click "Next". On the Additional Configuration page you can specify a security group, cache parameter group, and maintenance window for your cache cluster.

If you're launching your cache cluster in a VPC, the security group portion of the page will show the VPC security group and if you chose to launch your cache cluster outside of the VPC, it'll show a cache security group instead. As you can see, our cache cluster will be launched in a VPC and as you know, a VPC security group acts like a firewall and that controls network access to our cache clusters so we need to select one or more specific security groups to control the inbound rules. Now we choose the default group and in the next step we authorize the access by defining suitable inbound rules for this group. Also, it is possible to create a new VPC security group and assign it to our cluster. You can do this procedure even after launching a cache cluster. A cache parameter group acts as a container for engine configuration values and it can be applied to one or more cache clusters.

If you've created a custom cache cluster group and you want to use it, select it from the Cache Parameter Group box.

Otherwise, just proceed with the default one which AWS created for you. The Maintenance Window feature allows you to specify the time range UTC during which any scheduled maintenance activity such as software patching or pending cache cluster modifications you requested would occur. Scheduled maintenance activities occur infrequently, generally once every few months, and will be announced on the AWS forum two weeks prior to being scheduled. You can select your specific window or choose "No Preference" for that, as we do. All right, click on "Next". You can review the options for your cache cluster on the review page. If you need to make any changes just click "Previous" and make the changes and come back. When all the options are entered as you want them click "Launch Cache Cluster". A message displays stating that your cluster is being created and that the launch process will take a few minutes to complete. Then just click "Close" when the panel appears and your cache cluster is displayed in the list. It should have a "Creating" status until the creation is complete and your cache cluster is ready to use.

By default, network access to your cluster is limited to the user account that was used to create it. Before you can connect to a cluster from an Amazon EC2 instance, you have to authorize that EC2 instance to access the cluster. The steps required just depend on whether you launched your cluster in a VPC environment or not. As you saw, of course, our cluster was launched in a VPC so we can connect to our ElastiCache cluster only from an EC2 instance running in the same VPC security group. We'll need to grant network ingress to the cache cluster. To grant network ingress from a VPC security group to a cache cluster in Amazon VPC, open the console and from the Navigation list click on Security Group. You could reach this section from the Amazon EC2 console as well.

Here you can see the list of all security groups for which we use our different services. As you probably remember, we selected the default group for our cluster so we need to configure its inbound rules. In the Details pane, click "Inbound" then click on "Edit". Now by clicking on "Add Another Rule", we can create a new rule box. Select "Custom TCP Rule" from the type box and in the Port Range field enter the port number for your cluster node. This must be the same number that you specified when you launched the cache cluster. If you don't remember, for Memcached the default port is 11211 and for Redis the default port number is 6379.

In the Source field type all zeroes forward slash zero so that any Amazon EC2 instance that you launch within your VPC can connect to your ElastiCache nodes.

Then finally, click "Save". When you launch an EC2 instance in your VPC, that instance will be able to connect to your ElastiCache cluster. As we mentioned before, you can create a new security group and assign it to your EC2 instance whenever you want to otherwise access to your ElastiCache cluster. Once you've authorized access to your cache cluster it's in the available state and you can log in to an Amazon EC2 instance and connect to a node in the cluster. In order to do so, you must first determine the cache node endpoint. On the Cache Clusters page of the Amazon ElastiCache console click the name of a cluster. Then on the Details page for the cluster click on the Nodes tab. On the Nodes tab the endpoint of a cache node is there to use in the next step. Remember that the endpoint for your cache node isn't available until your cache node is in the available state. To connect to a Memcached cache node using Telnet, connect to your Amazon EC2 instance by using the connection utility of your choice. Note that you'll need to download and install the Telnet utility on your EC2 instance.

At the command prompt of your EC2 instance insert this command substituting the endpoint of your cache node for the one shown on the screen.

You'll see outputs similar to that that you can see on the screen. Now you're connected to a cache node and you can run Memcached commands, as you can see on the screen.

If you want to connect to a Redis cache node, you need to install the Redis-Cli program. So, connect to your Amazon EC2 instance using the connection utility of your choice. Note that before you build Redis-Cli you will need to download and install the new compiler collection, GCC, in your instance. The Redis-Cli utility is included in the Redis software distribution. At the command prompt of your EC2 instance enter these commands.

You can connect to your Redis cache node. Enter these command substituting the endpoint of your cache node for the one shown on the screen. You'll see a Redis command prompt similar to that on the screen. You're now connected to the cache node and you can run Redis commands.

As soon as your cluster becomes available, your build for each hour or partial hour that you keep the cluster running, even if the cluster is idle. When you no longer need that cluster you can delete it. To delete your cluster in the Amazon ElastiCache console, locate that cache cluster in your list of clusters on the Cache Clusters page and then click "Delete". ElastiCache begins deleting immediately. The status changes to deleting in the list. As soon as the cache cluster status changes to deleted you can stop incurring charges for that cache cluster.

About the Author

Computer Engineer and Cloud Expert