Introduction to AWS Billing and Cost Management Tools: Part 2 of 3

Cost management in the cloud is a never-ending exercise for operation managers. Fortunately, there are some tools which can help companies get started in the right direction. Let’s take a closer look at them.

In part 1 of our three-part series, we introduced some of the AWS billing and reporting tools. We saw how those tools can be used to understand infrastructure spending at a more granular level. Some of the reports can be loaded into third-party analytical tools for further analysis.
As companies start to analyze their spending patterns from billing reports, they start thinking about optimization. The big questions they try to answer include the following broad categories:

  • How can we keep our spending within a reasonable limit?
  • How can we start optimizing costs? Where do we start?
  • How can we forecast future spending?

In this part of the series, we will introduce some of the cost management tools already available from AWS that can help answer some of these important questions.

Creating Billing Alerts

One of the first things companies need to do after transitioning to AWS is enabling billing alerts. AWS billing alerts enable us to set a threshold for monthly spending and receive notifications when spending exceeds that threshold.

This is proactive cost monitoring. If there is a consistent pattern of billing alerts every month, it means either the threshold (and perhaps the infrastructure budget) is too low for the normal, expected workload, or there is a good argument for scaling back current resource consumption.
An early response to billing alerts means there are no nasty surprises at the end of the month when the company credit card is charged.

Billing alerts can be enabled from the AWS Billing and Cost Management console’s Preferences tab. Once enabled, it can’t be deactivated. In the image below, we have enabled this feature:
Enabling AWS Billing Alerts
Once enabled, you can create an alert from the AWS CloudWatch console. Billing alerts can be created only from the US-East (North Virginia) region; however its monitoring would include costs from resources running in every region. If you try to create a billing alert from CloudWatch in any other region, you will receive a message like this:
AWS Billing Alerts can be set from only US-East
In the image below, we are creating a billing alarm from the CloudWatch console in US-East region. As you can see,  billing alarms can be based on:

  • Estimated total charge for the current account.
  • Estimated total charge for one or more linked accounts.
  • Estimated charges for one or more AWS service consumed by the current account.
  • Estimated charges for one or more AWS service consumed by one or more linked accounts.

AWS Billing Metric types
Here, we are interested in the total estimated charge in USD for the current account. In the alert definition, we are setting up a threshold of US $1,000 and want CloudWatch to send us an alert when the current month’s spending goes above that limit.
AWS Billing Alert properties
We had to create an SNS topic called AWS_Billing_Notification first and confirm its subscription before creating the alarm. In this case, the SNS topic would send notifications to an e-mail subscription, which could be our accounts payable department.
Also, note how CloudWatch lets us perform an auto-scaling operation in response to the alarm. This can be useful for specific use cases. For example, if we are running a busy site during a particular campaign or promotion season, we could be using multiple servers in a farm to keep up with higher-than-normal traffic.
Not surprisingly, this would also mean extra spending. However, we would want to keep the spending within our limit as well. With billing alerts, we can use auto scaling actions where the number of servers in the farm would be reduced once the spending hits the US$ 1,000 mark.
Once set up, the monitoring is automatic. Every time the spending threshold is reached, an e-mail like the following would be sent to the address specified:

You are receiving this email because your  estimated charges are greater than the limit you set for the alarm "AWS-EstimatedCharges-Alarm" in AWS Account XXXXXXXXXXXX.
The alarm limit you set was $ 1000.00 USD. Your total estimated charges accrued for this billing period are currently $ 1020.26 USD as of Thursday 16 June, 2016 17:37:52 UTC. The actual charges you will be billed in this statement period may differ from the charges shown on this notification. For more information, view your estimated bill at: https://console.aws.amazon.com/billing/home#/bill?year=2016&month=6
More details about this alarm are provided below:
---
Amazon CloudWatch Alarm "AWS-EstimatedCharges-Alarm" in the US East - N. Virginia region has entered the ALARM state, because "Threshold Crossed: 1 datapoint (1020.26) was greater than or equal to the threshold (1000.0)." at "Thursday 16 June, 2016 17:37:52 UTC".
View this alarm in the AWS Management Console:
https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#s=Alarms&alarm=AWS-EstimatedCharges-Alarm
Alarm Details:
- Name:                      AWS-EstimatedCharges-Alarm
- Description:                CloudWatch Alarm for Estimated Charges
- State Change:              OK -> ALARM
- Reason for State Change:    Threshold Crossed: 1 datapoint (1020.26) was greater than or equal to the threshold (1000.0).
- Timestamp:                  Thursday 16 June, 2016 17:37:52 UTC
- AWS Account:                XXXXXXXXXXXX
Threshold:
- The alarm is in the ALARM state when the metric is GreaterThanOrEqualToThreshold 1000.00 for 21600 seconds.
Monitored Metric:
- MetricNamespace:            AWS/Billing
- MetricName:                EstimatedCharges
- Dimensions:                [Currency = USD]
- Period:                    21600 seconds
- Statistic:                  Maximum
- Unit:                      not specified
State Change Actions:
- OK:
- ALARM: [arn:aws:sns:us-east-1:XXXXXXXXXXXX:AWS_Billing_Notification]
- INSUFFICIENT_DATA:

Creating Budgets

Closely related to billing alerts are budgets. This is a comparatively new feature and also available from the Billing and Cost Management console. Budgets in AWS serve the same purpose as they do in our daily lives.
They’re a way to pre-allocate funding for your AWS spending and allow for monitoring the actual usage against the allocated value.

Like billing alerts, the budgeting feature needs to be enabled first (see below) and once enabled, you cannot turn it off.
Enabling Budget
In the image below, we are creating a budget for an AWS account.  The wizard is fairly simple:

  • We are budgeting for two specific services: namely, EC2 instances and EBS volumes.
  • We are budgeting for the next six months (June to December).
  • We have a budget of $1,500 per month to spend for these services.
  • We would like to be notified when our actual spending has gone 2% over our budgeted amount.
  • We would like to be notified when our forecasted spending goes 2% over our budgeted amount.
  • The alarm notification should be sent to our accounts payable department’s e-mail address (behind the scene, AWS creates a CloudWatch alarm for this).

Creating an AWS budget
Once the budget is created, it’s visible from the Budget console. Notice how the status bar gives a visual representation of current spending against the budgeted value.
An AWS budget
Individual AWS services apart, you can create budgets for the following categories:

  • Specific API operation(s) against an AWS service: for example, you can budget for S3 PutObject requests.
  • Availability Zone: For example, your primary data center in ap-southeast-2a should have more costs allocated than ap-southeast-2b and  ap-southeast-2c. You can budget for low spending in those two AZs.
  • Linked accounts: If each department or business unit in your company has a separate AWS account, you can link them all to a master account which pays the consolidated bill at the end of the month. Using this feature, you can set budgets for each linked account.
  • Purchase option: With this type of budget you can decide how much you want to spend on on-demand instances and reserved instances.
  • Tag: You can budget for resources with a specific tag. For example, if each resource used in your development environment has a type tag of “DEV”, you can use that tag in the budget. That way you can budget for your development infrastructure. Another example could be a tag which identifies the business owner of a resource, say the marketing department. Using that tag, you may want to budget for the marketing department’s resource usage during a campaign season.

A few more things to be mindful about budgets:

  • By default, only two budgets are allowed per account. In our opinion, this is fairly restrictive.
  • Once a budget is created, it can’t be modified, it can only be deleted. This is restrictive as well.
  • With notification options (actual or forecasted spending against budgeted value), AWS creates necessary CloudWatch alarms and associated SNS topics behind the scene. However, these alarms are subject to AWS account limits. By default, an AWS account can have ten alarms per month for free. If you already have a large number of alarms configured, the budget alarms may actually cost you extra charges.

Cost Allocation Tags

While budgets and alerts can help keep costs down, cost allocation tags enable IT departments to apportion the total AWS bill to various internal and external customers. The traditional IT shop usually hosts a number of resources for different costs centers (sales, marketing, finance, corporate apps, HR etc.), projects or teams (application developers, business analysts, service desk or training). Specific tags can be added to each of these resources to identify or associate them with specific business owner or purpose.
For example, an EC2 instance can have two tags: “Owner” and “Environment” with values of say, “HR” and “Production” respectively. When cost allocation tags are enabled, AWS will generate a cost allocation report with the total monthly bill broken down by the enabled tags.  This is a CSV file created in an S3 bucket. In this particular example, if both the “Owner” and “Environment” tags are enabled, the report can show what IT should be charging back HR for hosting their production server.
In the image below, we can see three tags are enabled for cost allocation report.
AWS Cost Allocation Tags

What’s Next?

We have now seen how to create a threshold and a budget for limiting our spending in the AWS cloud and how to proactively recoup infrastructure costs from customers consuming IT services.  As we will see in the next and final part of this series, there are more sophisticated tools available for identifying optimization opportunities and take advantage of price reductions from AWS.
Read part 3 now!

 

Avatar

Written by

Sadequl Hussain

Sadequl Hussain is an IT pro based in Sydney, Australia. He comes from a strong database administration backround and has more than 15 years of experience in development, database management, training, and technical writing. Sadequl also holds a number of vendor certifications, including one from AWS. He loves working with cloud technologies, NoSQL / Big Data databases, automation toolsets, open source technologies and Windows / Linux system administration. When he is not doing any of these, Sadequl loves to spend time with his young family.


Related Posts

Albert Qian
Albert Qian
— November 13, 2019

Advantages and Disadvantages of Microservices Architecture

What are microservices? Let's start our discussion by setting a foundation of what microservices are. Microservices are a way of breaking large software projects into loosely coupled modules, which communicate with each other through simple Application Programming Interfaces (APIs). ...

Read more
  • AWS
  • Docker
  • Kubernetes
  • Microservices
Nisar Ahmad
Nisar Ahmad
— November 12, 2019

Kubernetes Services: AWS vs. Azure vs. Google Cloud

Kubernetes is a popular open-source container orchestration platform that allows us to deploy and manage multi-container applications at scale. Businesses are rapidly adopting this revolutionary technology to modernize their applications. Cloud service providers — such as Amazon Web Ser...

Read more
  • AWS
  • Azure
  • Google Cloud
  • Kubernetes
Avatar
Stuart Scott
— October 31, 2019

AWS Internet of Things (IoT): The 3 Services You Need to Know

The Internet of Things (IoT) embeds technology into any physical thing to enable never-before-seen levels of connectivity. IoT is revolutionizing industries and creating many new market opportunities. Cloud services play an important role in enabling deployment of IoT solutions that min...

Read more
  • AWS
  • AWS IoT Events
  • AWS IoT SiteWise
  • AWS IoT Things Graph
  • IoT
Avatar
Cloud Academy Team
— October 23, 2019

Which Certifications Should I Get?

As we mentioned in an earlier post, the old AWS slogan, “Cloud is the new normal” is indeed a reality today. Really, cloud has been the new normal for a while now and getting credentials has become an increasingly effective way to quickly showcase your abilities to recruiters and compan...

Read more
  • AWS
  • Azure
  • Certifications
  • Cloud Computing
  • Google Cloud Platform
Valery Calderón Briz
Valery Calderón Briz
— October 22, 2019

How to Go Serverless Like a Pro

So, no servers? Yeah, I checked and there are definitely no servers. Well...the cloud service providers do need servers to host and run the code, but we don’t have to worry about it. Which operating system to use, how and when to run the instances, the scalability, and all the arch...

Read more
  • AWS
  • Lambda
  • Serverless
Avatar
Stuart Scott
— October 16, 2019

AWS Security: Bastion Hosts, NAT instances and VPC Peering

Effective security requires close control over your data and resources. Bastion hosts, NAT instances, and VPC peering can help you secure your AWS infrastructure. Welcome to part four of my AWS Security overview. In part three, we looked at network security at the subnet level. This ti...

Read more
  • AWS
Avatar
Sudhi Seshachala
— October 9, 2019

Top 13 Amazon Virtual Private Cloud (VPC) Best Practices

Amazon Virtual Private Cloud (VPC) brings a host of advantages to the table, including static private IP addresses, Elastic Network Interfaces, secure bastion host setup, DHCP options, Advanced Network Access Control, predictable internal IP ranges, VPN connectivity, movement of interna...

Read more
  • AWS
  • best practices
  • VPC
Avatar
Stuart Scott
— October 2, 2019

Big Changes to the AWS Certification Exams

With AWS re:Invent 2019 just around the corner, we can expect some early announcements to trickle through with upcoming features and services. However, AWS has just announced some big changes to their certification exams. So what’s changing and what’s new? There is a brand NEW ...

Read more
  • AWS
  • Certifications
Alisha Reyes
Alisha Reyes
— October 1, 2019

New on Cloud Academy: ITIL® 4, Microsoft 365 Tenant, Jenkins, TOGAF® 9.1, and more

At Cloud Academy, we're always striving to make improvements to our training platform. Based on your feedback, we released some new features to help make it easier for you to continue studying. These new features allow you to: Remove content from “Continue Studying” section Disc...

Read more
  • AWS
  • Azure
  • Google Cloud Platform
  • ITIL® 4
  • Jenkins
  • Microsoft 365 Tenant
  • New content
  • Product Feature
  • Python programming
  • TOGAF® 9.1
Avatar
Stuart Scott
— September 27, 2019

AWS Security Groups: Instance Level Security

Instance security requires that you fully understand AWS security groups, along with patching responsibility, key pairs, and various tenancy options. As a precursor to this post, you should have a thorough understanding of the AWS Shared Responsibility Model before moving onto discussi...

Read more
  • AWS
  • instance security
  • Security
  • security groups
Avatar
Jeremy Cook
— September 17, 2019

Cloud Migration Risks & Benefits

If you’re like most businesses, you already have at least one workload running in the cloud. However, that doesn’t mean that cloud migration is right for everyone. While cloud environments are generally scalable, reliable, and highly available, those won’t be the only considerations dri...

Read more
  • AWS
  • Azure
  • Cloud Migration
Joe Nemer
Joe Nemer
— September 12, 2019

Real-Time Application Monitoring with Amazon Kinesis

Amazon Kinesis is a real-time data streaming service that makes it easy to collect, process, and analyze data so you can get quick insights and react as fast as possible to new information.  With Amazon Kinesis you can ingest real-time data such as application logs, website clickstre...

Read more
  • amazon kinesis
  • AWS
  • Stream Analytics
  • Streaming data