Open Source Observability
Start course
2h 12m

This course covers the core learning objective to meet the requirements of the 'Architecting for Management & Governance in AWS - Level 2' skill

Learning Objectives:

  • Understand the different AWS management services available to monitor the performance of a solution
  • Apply Amazon CloudWatch monitoring contols to respond to system-wide performance changes
  • Apply AWS Config controls to manage compliance based upon business guidelines

As architectural patterns change and become more complex, from the monolith to microservices, observability has become more and more challenging. With the distributed nature of applications, you now have to manage 10 times more infrastructure, all potentially with different programming languages supported by different teams. 

With observability, this means teams might be using different SDKs or agents to instrument their applications.  Because of this, the data they collect from the agents or SDKs may not be in a standardized data format.

This leads to two big issues: 

  1. Running multiple agents can create extra costs, and can be frustrating for engineering teams

  2. And having to correlate data from all the sources can be very slow and time-consuming  

This is why the OpenTelemetry standard was created. OpenTelemetry states that instead of installing and running multiple SDKs and agents, you can instrument your applications just once. Then, you collect metrics and traces all in the same standardized format, and can then send that data to the monitoring platforms of your choice. This set of standards, which is part of the Cloud Native Computing Foundation, is becoming very popular with organizations that are implementing observability.  So much so that AWS decided to develop a distribution of the OpenTelemetry standard called AWS Distro for OpenTelemetry. 

The main benefit of using AWS’ distribution instead of running your own is that AWS ensures that it is predictable and secure through their own continuous testing. The other benefit is that if you run into any issues, you then have the ability to reach out to AWS Support for service-related help. You can use AWS Distro for OpenTelemetry on basically any type of compute, including ECS, EKS, Fargate, EC2, Lambda, and on-premises services. From there, you can instrument your applications with the AWS Distro for OpenTelemetry SDKS, agents, collectors, and exporters. 

But once you instrument your applications with AWS Distro for OpenTelemetry, where do you send that data? You have four main options: you can send the data to Amazon CloudWatch, AWS X-Ray, Amazon OpenSearch Service, and Amazon Managed Service for Prometheus.

Let’s talk about the last option, Amazon Managed Service for Prometheus. Prometheus is an open-source monitoring system that has become very popular for gathering metrics and creating alerts. It has become the go-to monitoring platform for a lot of containerized workloads - especially Kubernetes workloads. 

Because of this, Amazon has created its own compatible version of Prometheus called Amazon managed Service for Prometheus, or AMP. AMP is a metric-only service and does not support logging or tracing. Once you collect data using AWS Distro for OpenTelemetry, you can send the metrics from your application to AMP.  AMP is generally recommended in two cases:

  1. If you’re already using Prometheus and running it in AWS, you can switch to the managed service to reduce burden

  2. Or if you’re looking for a service that is better optimized to monitor your container workloads. 

Keep in mind that Prometheus monitoring data can be integrated with CloudWatch, so that you can query and alarm based on that data. 

Since Prometheus is used mainly as a reporting service, often more visualization of metrics is needed. This is where customers can choose to integrate another service. A popular open-source service for analysis and visualization is called Grafana.

You can host a Grafana server yourself the DIY way using Amazon EC2 or you can use the Amazon Managed Grafana service….which I’m sure you can guess through the name is AWS’ version of a managed Grafana server. With Amazon Managed Grafana, you can create dashboards and other visualizations to gain insight into your metrics, logs, and traces. With Amazon Managed Grafana, it lessens the burden of management, and you still get integration with the AWS landscape with the ability to use IAM, multi-accounts, SSO, CloudTrail, and more. 

So, if you have an open source requirement for observability and still want to use AWS, you have another option for an observability stack in AWS by using AWS Distro for OpenTelemetry for data collection, Prometheus for data reporting, and Grafana for data visualization. 

About the Author
Learning Paths

Stuart has been working within the IT industry for two decades covering a huge range of topic areas and technologies, from data center and network infrastructure design, to cloud architecture and implementation.

To date, Stuart has created 150+ courses relating to Cloud reaching over 180,000 students, mostly within the AWS category and with a heavy focus on security and compliance.

Stuart is a member of the AWS Community Builders Program for his contributions towards AWS.

He is AWS certified and accredited in addition to being a published author covering topics across the AWS landscape.

In January 2016 Stuart was awarded ‘Expert of the Year Award 2015’ from Experts Exchange for his knowledge share within cloud services to the community.

Stuart enjoys writing about cloud technologies and you will find many of his articles within our blog pages.