Introduction to AWS Billing and Cost Management Tools: Part 1 of 3

Amazon Web Services offers some great tools for managing, analyzing, forecasting, and optimizing infrastructure costs in the cloud. In this three-part blog series, we will explain how you and your organization can take advantage of them.

One of the major factors driving companies to adopt cloud solutions like Amazon Web Services is the minimal upfront cost of resource provisioning followed by an easy, pay-as-you-go operating model. These two factors, along with the disposable nature of the wide variety of available computing resources, and the ease of resizing those resources, makes AWS an attractive alternative to large data centers.
However, companies moving their workloads to the cloud will sooner or later start thinking about optimizing costs. Medium-to-large organizations making a transition to the cloud or who have completed a transition will want to see the benefits of their investment.

These companies – when running large server fleets in their own data centers – were accustomed to seeing computing and storage resources sitting idle and incurring costs even after a project or application’s life-cycle ended. Unfortunately, the practice often remains unchanged after they have migrated to the cloud. The result is unnecessary but easily avoidable expenditure.

Even small companies and cloud-native startups sometimes fall into this trap. Typically, these companies try to minimize their resource footprint and maximize the return on investment in infrastructure. But as the R&D phase ends and products enter the market, keeping infrastructure highly-available takes higher priority than running them optimally. Empty servers, underutilized RDS instances or unattached storage volumes are often overlooked in favor of day-to-day operations.

The outcome is the same: costs are often comparable, or even greater than running an on-premise network. After all, it’s not possible for cloud providers to know when a customer stops using a resource and shut it down for them. That responsibility still lies with the customer.

AWS Billing and Cost Management Tools

Fortunately, Amazon Web Services (AWS) and third-party vendors offer an array of tools for managing and monitoring cloud operating costs. Infrastructure and operation managers will find these tools extremely useful because they can help find:

  • What resource types are incurring the largest share of total cost (EC2, RDS, Redshift etc.).
  • The trend in monthly bills (is it the same, is it increasing or decreasing?).
  • A forecast for the current monthly bill.
  • A means to send alerts when costs exceed some threshold.
  • Potential areas of savings.
  • Estimated costs before a resource is provisioned.

Using the insight from these tools, infrastructure managers can decide:

  • Which AWS resources or service can be shut down or retired.
  • Which AWS resources can be resized based on their usage pattern.
  • How to apportion the total bill to various customers based on their resource usage.

In this three-part series, we will introduce some of these tools. We will start with how to look at the cost reports. We will then introduce some tools that can help better manage the resource footprint. Finally, we will talk about tools that can forecast costs.

Back to Basics

Let’s start with the basics of AWS billing management.
When you choose the Billing and Cost Management option from the AWS console drop down menu, the screen that appears gives an overall picture of the account’s current spending for the month. It also shows the costs incurred for last month’s usage, and based on current usage pattern this month’s billing forecast. Here is an example:
AWS Billing Summary
Another widget on the screen breaks down the current month’s spending by service type. As we can see from the image below, the main cost component for this billing cycle is the EC2 service:
AWS month-to-date spending by service

Making Sense of AWS Bills

To see the actual billing information, you can either choose to access it from the Bills console or you can refer to the PDF copy sent via e-mail. The PDF version is available if you have chosen the first option from billing preferences screen. Both options are shown below:
AWS billing console
AWS billing preferences
We find the latter method useful because it allows you to make the information accessible to relevant people without giving them access to the console. One other hand, the console version allows you to drill down the billing components by service and region, something the PDF version does not offer.
Also, it’s worthwhile to enable the last option (“Receive Billing Reports”). This allows billing reports to be delivered to an S3 bucket. The billing reports are delivered once or more daily, which allows you to refer to them whenever you need rather than waiting for the monthly invoice to come.
In the following image, we have accessed the latest bill from the Billing Management console. We can see that the majority of the costs is associated with Elastic Compute Cloud (EC2).
AWS bill summary view
Drilling down on the EC2 component by expanding the arrow beside it, we can see the where the money has been spent. It shows we have been running our EC2 instances in three regions (ap-southeast-2, us-east and us-west). It also shows individual costs incurred by EBS volumes, data transfers, and Elastic IPs.
AWS EC2 costs for Sydney region
AWS EC2 costs for North Virginia region
AWS EC2 costs for Oregon region
Going through these individual regions shows us North Virginia has the highest cost component for EC2 service. This is meaningful information, but as we will see later, there are better ways to find it.

AWS Cost Explorer

AWS Cost Explorer is a graphical cost analysis tool available from the Billing and Cost Management console’s menu. It uses the data from each month’s spending for up to the last thirteen months and can forecast a spend pattern for the next three months. The spend graphs can be tailored for certain time periods as well. With cost explorer, AWS customers can easily visualize where their money is being spent. It can show spending by:

  • AWS regions: This can be helpful for large, global companies running workload from multiple regions.
  • Availability Zone: Can be useful when you are using an AZ as a “backup” location and not expecting too much cost from there.
  • Service: Which type of service is costing most? Typically it’s EC2, but then there can be other resources like EMR or Redshift clusters.
  • EC2 instance types: Which EC2 instances are costing the most? Are they supposed to run with large configuration?
  • Linked account: If multiple AWS accounts are linked to a “parent” account paying the bill, this report can show which linked account is accruing highest cost. The customer can then be billed back if necessary.
  • API calls: What type of API calls are expensive?
  • Purchase options: How much is being spent for reserved instances? How much is spent for on-demand instances? If on-demand instances are using larger portion of the spending, it may be worthwhile to think about reserved instances.
  • Cost allocation tags: You can attach “tags” to AWS resources.  Costs analysis can be mapped against each of the tags.

The following image shows the cost explorer dashboard once it has been enabled. As you can see, AWS cost explorer comes with a few “canned” reports:
AWS Cost Explorer dashboard
In the image below, we are creating a custom view. This is for our RDS instance spending over the last three months and we want to predict how much we would be spending in the next three months. If you click on the image, it will open in a new browser tab.
Creating new AWS cost report for RDS
From the image, you can see we are providing the following details:

  • A name for the report.
  • A time range of the last three months as an analysis period.
  • A time range of the next three months for forecasting.
  • Filter the spend by service (in this case, RDS).
  • If we did not use a forecast window, we could further drill down by grouping. For example, we could group our last three months’ RDS spending by region.
  • A view of monthly expense instead of a daily breakdown.

From this particular report, we can see our RDS spending has been fairly steady in the last three months – and AWS also predicts a similar trend in the coming three months.
Once the report is ready, we can save it to use later. To access a cost explorer report you can choose it from the drop-down list in the Launch Cost Explorer console. In the same drop down list, there is an option for “View/Manage all reports.” Clicking on that option will show a list of both user-defined and pre-configured reports. You can delete your existing reports from that list:
Existing AWS cost reports

Cost and Usage Reports

AWS customers can also create custom reports based on their usage pattern. These reports are not like cost explorer reports with graphs and trends, but they offer greater granularity of information. The feature is accessible from the Reports tab of the Billing and Cost Management console.
Also, cost and usage reports are automatically generated once defined and periodically saved in an S3 bucket. The frequency of reports is also customizable (hourly or daily). Cost and usage reports are comma separated files (CSV) with a large number of columns. With large AWS footprints, these documents can be fairly large and almost unreadable. You can load them into third-party tools to further analyze, roll-up etc.
To get started, you can create a custom report by clicking on the “Create report” button. This will start a wizard where you can specify various options as shown below:
AWS Cost and Usage Report console
Note how we have to set proper permissions on the S3 bucket. The tool helps us by providing a sample bucket policy:
Creating AWS usage report
Configuring AWS usage report
Once you review and complete the Wizard, the report will be available:List of AWS usage reports
From the image, you can see a report’s definition can’t be modified. For example, if you want the report to go to a separate S3 bucket, you have to delete and recreate it.
There are some pre-built reports wizards like the EC2 Instance Usage Report. Clicking on this link from the console will open another window where you can provide custom search criteria. In the image below, we are trying to see how much we are spending every day for our m3.large instance types. We have divided the report by platform. From this report, you can see:

  • Most of our m3.large instance are using Red Hat Enterprise Linux, while some are using other forms of Linux and the remainder are Windows boxes.
  • The average daily spend is $500 per day, but it has fallen slightly in the last two days.

AWS Instance Usage Report
Similarly, we can go beyond the last 14 days and see if we are achieving any savings on EC2 over the last year. From the image below, it seems we are: in August 2015, this AWS account was spending $85,000 for its EC2 instances. As of June 2016, it is spending an average of $65,000, a cost reduction of $20,000:
AWS Instance Usage Report

Billing Reports

We will wrap up this post with an introduction to Billing Reports. Billing reports are yet another type of report and like cost and usage reports, they too have to be saved in an S3 bucket in CSV format. Billing Reports can be enabled from the AWS Billing preferences screen. Once the S3 bucket has been verified, you can choose to receive the following types of reports:

  • Monthly report
  • Detailed report
  • Cost allocation report
  • Detailed billing report with resources and tags

Key Takeaways

As you can see, the AWS bill at the end of the month gives only a high level picture of the last billing cycle. You can take advantage of different types of reports and analyze the historical data to better understand your spending pattern. Understanding the spending pattern is only half the job though. The next step should be using that information to optimize the costs. And this is what we will cover in our next article.

Click here to read Part 2 of this series!

 

Avatar

Written by

Sadequl Hussain

Sadequl Hussain is an IT pro based in Sydney, Australia. He comes from a strong database administration backround and has more than 15 years of experience in development, database management, training, and technical writing. Sadequl also holds a number of vendor certifications, including one from AWS. He loves working with cloud technologies, NoSQL / Big Data databases, automation toolsets, open source technologies and Windows / Linux system administration. When he is not doing any of these, Sadequl loves to spend time with his young family.


Related Posts

Joe Nemer
Joe Nemer
— September 15, 2020

New Content: Azure DP-100 Certification, Alibaba Cloud Certified Associate Prep, 13 Security Labs, and Much More

This past month our Content Team served up a heaping spoonful of new and updated content. Not only did our experts release the brand new Azure DP-100 Certification Learning Path, but they also created 18 new hands-on labs — and so much more! New content on Cloud Academy At any time, y...

Read more
  • AWS
  • Azure
  • DevOps
  • Google Cloud Platform
  • Machine Learning
  • programming
Joe Nemer
Joe Nemer
— August 28, 2020

AWS Certification Practice Exam: What to Expect from Test Questions

If you’re building applications on the AWS cloud or looking to get started in cloud computing, certification is a way to build deep knowledge in key services unique to the AWS platform. AWS currently offers 12 certifications that cover major cloud roles including Solutions Architect, De...

Read more
  • AWS
  • AWS Certifications
Patrick Navarro
Patrick Navarro
— August 25, 2020

Overcoming Unprecedented Business Challenges with AWS

From auto-scaling applications with high availability to video conferencing that’s used by everyone, every day —  cloud technology has never been more popular or in-demand. But what does this mean for experienced cloud professionals and the challenges they face as they carve out a new p...

Read more
  • AWS
  • Cloud Adoption
  • digital transformation
Avatar
Andrew Larkin
— August 18, 2020

Constant Content: Cloud Academy’s Q3 2020 Roadmap

Hello —  Andy Larkin here, VP of Content at Cloud Academy. I am pleased to release our roadmap for the next three months of 2020 — August through October. Let me walk you through the content we have planned for you and how this content can help you gain skills, get certified, and...

Read more
  • alibaba
  • AWS
  • Azure
  • content roadmap
  • Content updates
  • DevOps
  • GCP
  • Google Cloud
  • New content
Alisha Reyes
Alisha Reyes
— August 5, 2020

New Content: Alibaba, Azure AZ-303 and AZ-304, Site Reliability Engineering (SRE) Foundation, Python 3 Programming, 16 Hands-on Labs, and Much More

This month our Content Team did an amazing job at publishing and updating a ton of new content. Not only did our experts release the brand new AZ-303 and AZ-304 Certification Learning Paths, but they also created 16 new hands-on labs — and so much more! New content on Cloud Academy At...

Read more
  • AWS
  • Azure
  • DevOps
  • Google Cloud Platform
  • Machine Learning
  • programming
Alisha Reyes
Alisha Reyes
— July 16, 2020

Blog Digest: Which Certifications Should I Get?, The 12 Microsoft Azure Certifications, 6 Ways to Prevent a Data Breach, and More

This month, we were excited to announce that Cloud Academy was recognized in the G2 Summer 2020 reports! These reports highlight the top-rated solutions in the industry, as chosen by the source that matters most: customers. We're grateful to have been nominated as a High Performer in se...

Read more
  • AWS
  • Azure
  • blog digest
  • Certifications
  • Cloud Academy
  • OWASP
  • OWASP Top 10
  • Security
  • VPCs
Avatar
Cloud Academy Team
— July 9, 2020

Which Certifications Should I Get?

The old AWS slogan, “Cloud is the new normal” is indeed a reality today. Really, cloud has been the new normal for a while now and getting credentials has become an increasingly effective way to quickly showcase your abilities to recruiters and companies. With all that in mind, the s...

Read more
  • AWS
  • Azure
  • Certifications
  • Cloud Computing
  • Google Cloud Platform
Alisha Reyes
Alisha Reyes
— July 2, 2020

New Content: AWS, Azure, Typescript, Java, Docker, 13 New Labs, and Much More

This month, our Content Team released a whopping 13 new labs in real cloud environments! If you haven't tried out our labs, you might not understand why we think that number is so impressive. Our labs are not “simulated” experiences — they are real cloud environments using accounts on A...

Read more
  • AWS
  • Azure
  • DevOps
  • Google Cloud Platform
  • Machine Learning
  • programming
Joe Nemer
Joe Nemer
— June 19, 2020

Kickstart Your Tech Training With a Free Week on Cloud Academy

Are you looking to make a jump in your technical career? Want to get trained or certified on AWS, Azure, Google Cloud Platform, DevOps, Kubernetes, Python, or another in-demand skill? Then you'll want to mark your calendar. Starting Monday, June 22 at 12:00 a.m. PDT (3:00 a.m. EDT), ...

Read more
  • AWS
  • Azure
  • cloud academy content
  • complimentary access
  • GCP
  • on the house
Alisha Reyes
Alisha Reyes
— June 11, 2020

New Content: AZ-500 and AZ-400 Updates, 3 Google Professional Exam Preps, Practical ML Learning Path, C# Programming, and More

This month, our Content Team released tons of new content and labs in real cloud environments. Not only that, but we introduced our very first highly interactive "Office Hours" webinar. This webinar, Acing the AWS Solutions Architect Associate Certification, started with a quick overvie...

Read more
  • AWS
  • Azure
  • DevOps
  • Google Cloud Platform
  • Machine Learning
  • programming
Rebecca Willis
Rebecca Willis
— June 3, 2020

Azure vs. AWS: Which Certification Provides the Brighter Future?

More and more companies are using cloud services, prompting more and more people to switch their current IT position to something cloud-related. The problem is most people only have that much time after work to learn new technologies, and there are plenty of cloud services that you can ...

Read more
  • AWS
  • Azure
  • certification
Alisha Reyes
Alisha Reyes
— June 2, 2020

Blog Digest: 5 Reasons to Get AWS Certified, OWASP Top 10, Getting Started with VPCs, Top 10 Soft Skills, and More

Thank you for being a valued member of our community! We recently sent out a short survey to understand what type of content you would like us to add to Cloud Academy, and we want to thank everyone who gave us their input. If you would like to complete the survey, it's not too late. It ...

Read more
  • AWS
  • Azure
  • blog digest
  • Certifications
  • Cloud Academy
  • OWASP
  • OWASP Top 10
  • Security
  • VPCs