Visualizing Costs in AWS
This course explores how to visualize your costs in AWS and share them with others. We'll look at some of the tools you can use for this purpose, including the AWS Billing Service, AWS Cost and Usage Reports (CURs), AWS Glue, Amazon Athena, and Amazon QuickSight. You'll follow with guided demonstrations from the AWS platform showing you how to use these services.
- Learn how to visualize your AWS costs
- Get a practical understanding of how to use the AWS Billing Service, AWS Cost and Usage Reports (CURs), AWS Glue, Amazon Athena, and Amazon QuickSight
This course is intended for anyone who wants to understand the tools available for visualizing costs in AWS
This course is beginner-friendly, however, to get the most out of it, it would be useful to have some knowledge of cost optimization theory, cloud & finance-related terminology, and a basic understanding of AWS Athena, AWS Glue, Amazon Quicksight, and SQL commands.
Amazon QuickSight lets you easily visualize and analyze your data. Similar to Athena, you can put datasets from AWS and outside sources into the service, and let it process the data for you for an easier exploration and deeper analysis. What's more is that you can create outstandingly detailed and precise visualizations. Moreover, the service that's you publish and share your dashboards and analyses with anyone, regardless of the receiver's AWS knowledge. The only condition is having an access to the same AWS Cloud account.
Additionally, it is possible to simply share the analysis dashboards by printing them out on paper or sending them as PDF files by email. QuickSight makes it easily accessible for everyone, which helps you communicate your numbers to whoever needs them.
The best way to use QuickSight is shown in our illustration right here, which you can find also in the official AWS Documentation. Starting from the lower left side, you basically always start by gathering together all your data, be it from your AWS environment or from some other source. The next step is preparing your data for the actual analysis. Then it takes the examination of your analysis and if necessary shares the dashboard with others. And finally, the results will help you to make better decisions based on your findings taken from the analysis.
So basically the idea with QuickSight is to access data and prepare it for reporting and better decision making. When you create an analysis, the typical workflow in theory looks like the following.
So let me explain it in five steps. So number one, create a new analysis, obviously it will be empty. Step two, add new or existing datasets. Step three, choose fields to create the first chart. QuickSight will automatically suggest the best visualizations for you, but you can change it later on. Step four, add more charts, tables, or insights to the analysis. You can resize and rearrange them on one or more sheets. Use extended features to add variables, custom controls, colors, themes, additional pages called sheets and many other options. And the last step, step five, is to publish your analysis as a dashboard and share it with other people.
All right. So let me just get started by showing you how to create a new analysis and we will just work through these five steps, so you get a better understanding. Okay, let's start with steps one and two. So first we go to Amazon QuickSight by simply typing the name into the search field. What we want to do is create a new analysis. So here I have a few of my old analysis and a few default ones, but what I want to do is create a new one. So here I choose my dataset. It's the OG test dataset that we created when we were about to use Athena.
Alright, and this is the QuickSight dashboard. From here on, we will start to apply filters and create our visualization and prepare our dashboard.
Okay, now that we have our blank analysis, let's apply some filters to it. The filter is basically a sort of query that is predefined for QuickSight. And by applying a filter, I'll choose the unblended cost, which will show me the value of this particular part of my dataset. Easy so far, right?
Okay. The next thing. I want to bring a little bit more depth into my analysis. So let's add another filter, which will be Line Item Usage Account ID. By combining these two, it reveals how much of the total unblended costs belong to which account exactly. In my particular case, only one account actually incurs some expenses. Seeing the total costs might be good, but it doesn't really help me.
So the next filter I want to apply is the Line Item Product Code. This will show me the exact name of the service that caused an expense. On the right side, now you can see a list of all the service names that can be found in our dataset. By applying these three filters, we build a graph for ourselves. Although it seems a little bit confusing still when looking at the x and y axes with all those Line Item terms.
So the next thing I want to do is change the names of the axes. By clicking on the configuration button on the left-hand side, the format visual sidebar opens up, and here simply select y-axis first and let's type in our name for the y-axis, which should be Linked Account ID. And next is the same thing, but with the x-axis, just expand the tab and type in cost. And now it should be much easier to read. Basically we have the cost by Linked Account ID.
Let's edit the title of our visual as well into something easier, maybe Cost by account and service, that seems fitting to me. Additionally, you can make some small adjustments to make it easier to read, or maybe just to appeal more to your tastes. By hovering with the mouse over the single objects, you can get more detailed information.
Remember if you share your visualization as an email or printed out version, you won't be able to see that information though. When clicking on the title of the axis, you can expand to another menu. Here you can sort, choose between the format or rename the title, or even apply other filters. Eventually this may become handy for you, so keep that in mind.
If you feel adventurous, you can even adjust the theme of your visualization. Let's add another visual now. Go to the upper left corner, click on the plus button and click on Add visual. This gives us another blank visualization, which we will fill up with other fields and filters. For this particular one, I thought it would be nice to know about how many hours of usage we actually have been working with our services.
So for this, I will choose the Item Line Usage Amount and combine it with a Start date. Now we can tell how much and how frequent I used to just been over the past year.
Next, I will show you how to add a calculated field, which is basically the filter that we're using for our visualization. The field will be called Purchase option and will show us the purchase type for each easy two instance that we have been using. This means we will visualize how much we spent on instances that we used on demand or reserved instances. And even more interestingly, how many of those for the cheapest spot instances.
As a little reminder, Spot Instances are those disposable and temporary instances that are only available for a limited time. They are much cheaper, but you can't really use them in the longterm.
So anyway, the formula we're going to use is one that I've borrowed from wellarchitectedlabs.com. The formula logic is as follows. If the first part of the Line Item usage type equals spot usage, then the purchase option will be shown as a spot. Otherwise, if the product usage type is box usage, then the purchase option will be shown as pricing term. And if none of them are true, then the purchase option will simply be shown as other.
For this logic the formula uses two IF statements. The first IF statement compares to the string located in the field of the Line Item Usage Type to the term Spot Usage or Spot. The second IF statement compares the string in product usage type to box usage. And if both IF statements turn out to be false, the output will simply be this string other.
Okay, now that this is done, let's apply our new filter and see what's happening. On the right side, there is now a list with the Purchase options themselves. Now there's this huge orange line crossing our graph that doesn't really add any information. So let's get rid of it.
For this, we will add a custom filter. So click on filter, search for the Purchase option, select it. And the filter type, select a custom filter list, select exclude, and type in other, which is the term that equals the name of the line that we want to get rid of in our graph. Finally, simply click on Apply, and you see right away the tiny little bit that displayed our on-demand usage is now clearly visible to us.
Now, additionally, let's exclude the empty line as well. And we are left with only our on-demand line, which if you remember our Purchase option filter proves that we never used any spot instances in fact, which is a shame, but never mind.
Heinrich Marks and is a FinOps Consultant at kreuzwerker Frankfurt GmbH, Germany. His expertise lies in AWS cost optimization and FinOps methodology at the enterprise level.
At kreuzwerker, we focus on AWS, Software Engineering, and Cloud Financial Management. We've faced many cases with our customers where costs in the cloud got out of hand and lost touch with the actual business values. This made us work harder on solutions and find ways to master financial management in the cloud.
Over time, we have already been able to save our customers many millions of dollars. Moreover, we are also proud partners of the FinOps Foundation and the leading professionals in Germany. Today, we want to share this knowledge with you and spread more awareness on the topic!