(Mis)managing your AWS Costs
We all know that AWS lets us deploy applications in ways that would have been unimaginable using traditional IT infrastructures and processes. But for that very reason it’s a bit too easy to lose track of exactly what you have running at any one time. The end of a billing period might consequently bring unexpected AWS costs indicating that your ongoing charges have soared out of control.
To avoid this, you’ll have to better understand where these surprises can come from. In this article I will describe what I believe to be the five biggest AWS costs control and resources management mistakes people and organizations currently make.
1. Using oversized instances, too many instances, or leaving instances running idle
Carefully choosing the correct instance type is a fundamental decision you’ll have to confront. What size instances should you run? How many will you need? How will you keep track of all your resources (a good place to start is your AWS account’s Billing page)? Each infrastructure deployment has it’s own unique characteristics, so these are questions that only you and your organization can answer. Instance types that are specifically optimized for compute, memory, or storage are obviously going to cost more than low-tier or general versions. It’s your responsibility to ensure you’re not over-provisioning for your specific project. But remember: using oversized instances will cost you money, using too many instances will cost you money, and leaving unused instances running will, you guessed it, cost you money.
2. Choosing the wrong instance model
Besides instance types, there are also three three distinct ways to purchase instance resources and thereby closely control your AWS costs:
- On-Demand Instances allow you to pay a fixed rate by the hour with no commitment.
- Reserved Instances provide a capacity reservation, offering a significant discount over hourly on-demand prices when you commit to longer-term instance purchases.
- Spot Instances enable you to bid your own price for instance capacity. Assuming that your applications have flexible start and end times and can survive service interruptions when instances at your bid price aren’t unavailable, this can provide significant savings.
For a first time AWS user, On-Demand Instances are the easiest to deploy and “no-commitment” sounds attractive. But using On-Demand for any sustained periods can generate unexpected AWS costs at the end of the month. Continuously stopping and starting these instances will also prove costly. If you are planning on deploying a site that needs to operate 24/7 and expect to run this site over for the next 12 months, you would be far better off with a reserved instance, since you’ll know your exact costs up front. If you’re planning to deploy infrastructure that’s a little more complex and requires Elastic Load Balancers and Auto Scaling, you will probably need to draw on a combination of both On-Demand and Spot Instances, carefully calculating the relative AWS costs. Doing this right requires some careful planning.
3. Taking too many snapshots
You can back up the data on your Amazon EBS volumes to Amazon S3 by taking point-in-time snapshots. Backing up your data is not only a smart thing to do but, more often than not, it’s critical. However, EBS snapshots should be saved in moderation, or you may find yourself suffering from snapshot sprawl…which can exponentially increase your storage costs. Make sure you have an EBS Snapshots retention strategy in place (using S3 Lifecyle rules) that’s a good fit for your particular needs.
4. Not releasing allocated Elastic IPs
AWS allows you one Elastic IP (EIP) address associated with a running instance at no charge. However, you will be charged for each additional EIP that you associate with that instance per hour on a pro rata basis. To ensure efficient use of Elastic IP addresses (which are, by definition, a limited resource), AWS also imposes a small hourly charge when these IP addresses are not associated with a running instance or when they are associated with a stopped instance or an unattached network interface. That’s right: you may just stop your instance and assume that the IP associated with it will be automatically released. Unfortunately, this is not the case, and your accrued AWS costs will prove it. So make sure that you aren’t maintaining (and paying for) any unused EIPs.
5. Not monitoring AWS costs using CloudWatch and Smart Alerts
Amazon CloudWatch is a monitoring service for AWS cloud resources and the applications you run on AWS. You can use Amazon CloudWatch to collect and track metrics, monitor log files, and set alarms. You can set alarms on any of your metrics to receive notifications or automatically initiate other actions (like shutting down unused or underused EC2 instances) when a metric crosses your specified threshold. You can also use Auto Scaling to add or remove Amazon EC2 instances dynamically based on your Amazon CloudWatch metrics. If you have complex infrastructure of any kind running on AWS, you really need to use CloudWatch to keep up with it.
Most importantly, however, can be the Billing Alerts you create to warn you when your monthly costs reach a certain predetermined threshold.
Control AWS Costs: plan ahead
Careful consideration of the above points will hopefully help you keep your ongoing AWS costs down. As we mentioned, Amazon is a fabulous platform for deploying large applications at prices unimaginable until recently. However if these applications aren’t carefully monitored and kept in check, you may find that the money you’d hoped to save by deploying your infrastructure on AWS is silently flying out the window.
Read also EC2 Pricing: understanding compute costs on AWS
WaitCondition Controls the Pace of AWS CloudFormation Templates
AWS's WaitCondition can be used with CloudFormation templates to ensure required resources are running.As you may already be aware, AWS CloudFormation is used for infrastructure automation by allowing you to write JSON templates to automatically install, configure, and bootstrap your ...
The 9 AWS Certifications: Which is Right for You and Your Team?
As companies increasingly shift workloads to the public cloud, cloud computing has moved from a nice-to-have to a core competency in the enterprise. This shift requires a new set of skills to design, deploy, and manage applications in the cloud.As the market leader and most mature p...
Two New EC2 Instance Types Announced at AWS re:Invent 2018 – Monday Night Live
The announcements at re:Invent just keep on coming! Let’s look at what benefits these two new EC2 instance types offer and how these two new instances could be of benefit to you. If you're not too familiar with Amazon EC2, you might want to familiarize yourself by creating your first Am...
Google Cloud Certification: Preparation and Prerequisites
Google Cloud Platform (GCP) has evolved from being a niche player to a serious competitor to Amazon Web Services and Microsoft Azure. In 2018, research firm Gartner placed Google in the Leaders quadrant in its Magic Quadrant for Cloud Infrastructure as a Service for the first time. In t...
Understanding AWS VPC Egress Filtering Methods
In order to understand AWS VPC egress filtering methods, you first need to understand that security on AWS is governed by a shared responsibility model where both vendor and subscriber have various operational responsibilities. AWS assumes responsibility for the underlying infrastructur...
S3 FTP: Build a Reliable and Inexpensive FTP Server Using Amazon’s S3
Is it possible to create an S3 FTP file backup/transfer solution, minimizing associated file storage and capacity planning administration headache?FTP (File Transfer Protocol) is a fast and convenient way to transfer large files over the Internet. You might, at some point, have conf...
Microservices Architecture: Advantages and Drawbacks
Microservices are a way of breaking large software projects into loosely coupled modules, which communicate with each other through simple Application Programming Interfaces (APIs).Microservices have become increasingly popular over the past few years. The modular architectural style,...
What Are Best Practices for Tagging AWS Resources?
There are many use cases for tags, but what are the best practices for tagging AWS resources? In order for your organization to effectively manage resources (and your monthly AWS bill), you need to implement and adopt a thoughtful tagging strategy that makes sense for your business. The...
How to Optimize Amazon S3 Performance
Amazon S3 is the most common storage options for many organizations, being object storage it is used for a wide variety of data types, from the smallest objects to huge datasets. All in all, Amazon S3 is a great service to store a wide scope of data types in a highly available and resil...
How to Optimize Cloud Costs with Spot Instances: New on Cloud Academy
One of the main promises of cloud computing is access to nearly endless capacity. However, it doesn’t come cheap. With the introduction of Spot Instances for Amazon Web Services’ Elastic Compute Cloud (AWS EC2) in 2009, spot instances have been a way for major cloud providers to sell sp...
What are the Benefits of Machine Learning in the Cloud?
A Comparison of Machine Learning Services on AWS, Azure, and Google CloudArtificial intelligence and machine learning are steadily making their way into enterprise applications in areas such as customer support, fraud detection, and business intelligence. There is every reason to beli...
How to Use AWS CLI
The AWS Command Line Interface (CLI) is for managing your AWS services from a terminal session on your own client, allowing you to control and configure multiple AWS services.So you’ve been using AWS for awhile and finally feel comfortable clicking your way through all the services....