Course Introduction & Intro to Logging
Google Cloud Logging
The course is part of these learning paths
Logging is very important today given the volume and variety of data we deal with across different customer use cases. This course will enable you to take a more proactive approach towards identifying faults and crashes in your applications through the effective use of Google Cloud Logging. As a result, you will learn how to delegate the operational overhead to GCP through automated logging tools, resulting in a more productive operational pipeline for your team and organization.
Through this course, you will equip yourself with the required skills for streaming log data to Google Cloud Logging service and use metrics to understand your system's behavior. The course will start with an introduction to the Cloud Logging Service and then demonstrate how to stream logs using Cloud Logging Agent and the Python client library.
To get the most out of this course, you should already have an understanding of application logging.
This course is suited for anyone interested in logging using Google Cloud Platform (GCP) Cloud Logging.
- Source code for this course: https://github.com/cloudacademy/Managing-Application-Logs-and-Metrics-on-GCP
- Google Cloud fluentd source code: https://github.com/GoogleCloudPlatform/google-fluentd
- Google Cloud fluentd additional configurations: https://github.com/GoogleCloudPlatform/fluentd-catch-all-config
- Google Cloud fluentd output plugin configuration: https://cloud.google.com/logging/docs/agent/logging/configuration#cloud-fluentd-config
- Package release history: https://pypi.org/project/google-cloud-logging/#history
- Metrics Explorer pricing: https://cloud.google.com/stackdriver/pricing#metrics-chargeable
Welcome, in this lecture, we will learn about the importance of logging. In today's competitive era, you want your systems to be up and running all the time with minimal disruption, no unintentional behavior, and in a secure way. This is a very nice wishlist, or I can say an ideal goal, but reality could be different.
When you run complex systems at scale, which can involve many different components and many moving parts, things can go wrong or behave in unexpected ways. For example, if your infrastructure compromises thousands of servers, then there is a high chance of a disk failure or server crash. Or if you have an application that has several features and frequent changes, some functionality can behave differently with the new code change.
So in reality, failures are bound to happen. Now the question is, how do we determine which part of the system failed or is not behaving as expected? The answer to the above question is logging. Logging is a critical part of any system or an application and it should be a first-class citizen when developing software. Logging helps system administrators, application developers, and the support team save countless hours when trying to understand the behavior of an application or events happening behind the scenes.
Besides the operational benefit, logging can have many hidden benefit and can add new business value. By analyzing logs, one can unlock different use-cases like how users are interacting with the application, which feature is most used or widely in demand, et cetera. So, logging seems to be an amazing way to enhance the application, but many question comes to mind.
Where do I store the logs? How do I search my logs for debug purposes? How do I get value out of the logs? Do I need to write my own logging tool or framework? Are there any tools available in open source that can be used? How do I manage the logs?
Logging is an essential part of the application, but managing logs can be challenging. To simplify logging, we can use the Google Cloud Platform offering, Cloud Logging. GCP Cloud Logging simplifies log management and log analysis. We will learn more about GCP Cloud Logging in the next lecture.
Pradeep Bhadani is an IT Consultant with over nine years of experience and holds various certifications related to AWS, GCP, and HashiCorp. He is recognized as HashiCorp Ambassador and GDE (Google Developers Expert) in Cloud for his knowledge and contribution to the community.
He has extensive experience in building data platforms on the cloud and as well as on-premises through the use of DevOps strategies & automation. Pradeep is skilled at delivering technical concepts helping teams and individuals to upskill on the latest technologies.