In part 1 of our three-part series, we introduced some of the AWS billing and reporting tools. We saw how those tools can be used to understand infrastructure spending at a more granular level. Some of the reports can be loaded into third-party analytical tools for further analysis.
As companies start to analyze their spending patterns from billing reports, they start thinking about optimization. The big questions they try to answer include the following broad categories:
In this part of the series, we will introduce some of the cost management tools already available from AWS that can help answer some of these important questions.
One of the first things companies need to do after transitioning to AWS is enabling billing alerts. AWS billing alerts enable us to set a threshold for monthly spending and receive notifications when spending exceeds that threshold.
This is proactive cost monitoring. If there is a consistent pattern of billing alerts every month, it means either the threshold (and perhaps the infrastructure budget) is too low for the normal, expected workload, or there is a good argument for scaling back current resource consumption.
An early response to billing alerts means there are no nasty surprises at the end of the month when the company credit card is charged.
Billing alerts can be enabled from the AWS Billing and Cost Management console’s Preferences tab. Once enabled, it can’t be deactivated. In the image below, we have enabled this feature:
Once enabled, you can create an alert from the AWS CloudWatch console. Billing alerts can be created only from the US-East (North Virginia) region; however its monitoring would include costs from resources running in every region. If you try to create a billing alert from CloudWatch in any other region, you will receive a message like this:
In the image below, we are creating a billing alarm from the CloudWatch console in US-East region. As you can see, billing alarms can be based on:
Here, we are interested in the total estimated charge in USD for the current account. In the alert definition, we are setting up a threshold of US $1,000 and want CloudWatch to send us an alert when the current month’s spending goes above that limit.
We had to create an SNS topic called AWS_Billing_Notification first and confirm its subscription before creating the alarm. In this case, the SNS topic would send notifications to an e-mail subscription, which could be our accounts payable department.
Also, note how CloudWatch lets us perform an auto-scaling operation in response to the alarm. This can be useful for specific use cases. For example, if we are running a busy site during a particular campaign or promotion season, we could be using multiple servers in a farm to keep up with higher-than-normal traffic.
Not surprisingly, this would also mean extra spending. However, we would want to keep the spending within our limit as well. With billing alerts, we can use auto scaling actions where the number of servers in the farm would be reduced once the spending hits the US$ 1,000 mark.
Once set up, the monitoring is automatic. Every time the spending threshold is reached, an e-mail like the following would be sent to the address specified:
You are receiving this email because your estimated charges are greater than the limit you set for the alarm "AWS-EstimatedCharges-Alarm" in AWS Account XXXXXXXXXXXX. The alarm limit you set was $ 1000.00 USD. Your total estimated charges accrued for this billing period are currently $ 1020.26 USD as of Thursday 16 June, 2016 17:37:52 UTC. The actual charges you will be billed in this statement period may differ from the charges shown on this notification. For more information, view your estimated bill at: https://console.aws.amazon.com/billing/home#/bill?year=2016&month=6 More details about this alarm are provided below: --- Amazon CloudWatch Alarm "AWS-EstimatedCharges-Alarm" in the US East - N. Virginia region has entered the ALARM state, because "Threshold Crossed: 1 datapoint (1020.26) was greater than or equal to the threshold (1000.0)." at "Thursday 16 June, 2016 17:37:52 UTC". View this alarm in the AWS Management Console: https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#s=Alarms&alarm=AWS-EstimatedCharges-Alarm Alarm Details: - Name: AWS-EstimatedCharges-Alarm - Description: CloudWatch Alarm for Estimated Charges - State Change: OK -> ALARM - Reason for State Change: Threshold Crossed: 1 datapoint (1020.26) was greater than or equal to the threshold (1000.0). - Timestamp: Thursday 16 June, 2016 17:37:52 UTC - AWS Account: XXXXXXXXXXXX Threshold: - The alarm is in the ALARM state when the metric is GreaterThanOrEqualToThreshold 1000.00 for 21600 seconds. Monitored Metric: - MetricNamespace: AWS/Billing - MetricName: EstimatedCharges - Dimensions: [Currency = USD] - Period: 21600 seconds - Statistic: Maximum - Unit: not specified State Change Actions: - OK: - ALARM: [arn:aws:sns:us-east-1:XXXXXXXXXXXX:AWS_Billing_Notification] - INSUFFICIENT_DATA:
Closely related to billing alerts are budgets. This is a comparatively new feature and also available from the Billing and Cost Management console. Budgets in AWS serve the same purpose as they do in our daily lives.
They’re a way to pre-allocate funding for your AWS spending and allow for monitoring the actual usage against the allocated value.
Like billing alerts, the budgeting feature needs to be enabled first (see below) and once enabled, you cannot turn it off.
In the image below, we are creating a budget for an AWS account. The wizard is fairly simple:
Once the budget is created, it’s visible from the Budget console. Notice how the status bar gives a visual representation of current spending against the budgeted value.
Individual AWS services apart, you can create budgets for the following categories:
A few more things to be mindful about budgets:
While budgets and alerts can help keep costs down, cost allocation tags enable IT departments to apportion the total AWS bill to various internal and external customers. The traditional IT shop usually hosts a number of resources for different costs centers (sales, marketing, finance, corporate apps, HR etc.), projects or teams (application developers, business analysts, service desk or training). Specific tags can be added to each of these resources to identify or associate them with specific business owner or purpose.
For example, an EC2 instance can have two tags: “Owner” and “Environment” with values of say, “HR” and “Production” respectively. When cost allocation tags are enabled, AWS will generate a cost allocation report with the total monthly bill broken down by the enabled tags. This is a CSV file created in an S3 bucket. In this particular example, if both the “Owner” and “Environment” tags are enabled, the report can show what IT should be charging back HR for hosting their production server.
In the image below, we can see three tags are enabled for cost allocation report.
We have now seen how to create a threshold and a budget for limiting our spending in the AWS cloud and how to proactively recoup infrastructure costs from customers consuming IT services. As we will see in the next and final part of this series, there are more sophisticated tools available for identifying optimization opportunities and take advantage of price reductions from AWS.
Read part 3 now!
It's Flash Sale time! Get 50% off your first year with Cloud Academy: all access to AWS, Azure, and Cloud…
In this blog post, we're going to answer some questions you might have about the new AWS Certified Data Engineer…
This is my 3rd and final post of this series ‘Navigating the Vocabulary of Gen AI’. If you would like…