Course Introduction & Intro to Logging
Google Cloud Logging
Log Collection
Metrics Explorer
Summary
The course is part of these learning paths
See 2 moreLogging is very important today given the volume and variety of data we deal with across different customer use cases. This course will enable you to take a more proactive approach towards identifying faults and crashes in your applications through the effective use of Google Cloud Logging. As a result, you will learn how to delegate the operational overhead to GCP through automated logging tools, resulting in a more productive operational pipeline for your team and organization.
Learning Objectives
Through this course, you will equip yourself with the required skills for streaming log data to Google Cloud Logging service and use metrics to understand your system's behavior. The course will start with an introduction to the Cloud Logging Service and then demonstrate how to stream logs using Cloud Logging Agent and the Python client library.
Prerequisites
To get the most out of this course, you should already have an understanding of application logging.
Intended Audience
This course is suited for anyone interested in logging using Google Cloud Platform (GCP) Cloud Logging.
Resources
- Source code for this course: https://github.com/cloudacademy/Managing-Application-Logs-and-Metrics-on-GCP
- Google Cloud fluentd source code: https://github.com/GoogleCloudPlatform/google-fluentd
- Google Cloud fluentd additional configurations: https://github.com/GoogleCloudPlatform/fluentd-catch-all-config
- Google Cloud fluentd output plugin configuration: https://cloud.google.com/logging/docs/agent/logging/configuration#cloud-fluentd-config
- Package release history: https://pypi.org/project/google-cloud-logging/#history
- Metrics Explorer pricing: https://cloud.google.com/stackdriver/pricing#metrics-chargeable
So far, we have learned about what a Cloud Logging Agent can do. Now, let's learn how it export logs through the Google Cloud Logging. This diagram illustrates the flow of log data to Cloud Logging. An application write its log to disk and the Cloud Logging Agent reads those entries using the default or a custom input plugin. The Cloud Logging Agent adds some additional fields such as "Tag", "Timestamp", "Severity" and "Labels" to the log entry and that becomes a Log record.
The Log record is then forwarded to the filter plugin, which is used to remove any unnecessary records. The output plugin in our case is "fluentd-plugin-google-cloud" that sends the data to the Cloud Logging service. To ensure the log data flows to the Cloud Logging Service, Cloud Logging Agent requires access to OAuth2 token server , and the Logging API . If you're on Linux, it also requires access to the Google Cloud package repository at packages.cloud.google.com. If you're on Windows, it requires access to the Google Download Server at dl.google.com.
Pradeep Bhadani is an IT Consultant with over nine years of experience and holds various certifications related to AWS, GCP, and HashiCorp. He is recognized as HashiCorp Ambassador and GDE (Google Developers Expert) in Cloud for his knowledge and contribution to the community.
He has extensive experience in building data platforms on the cloud and as well as on-premises through the use of DevOps strategies & automation. Pradeep is skilled at delivering technical concepts helping teams and individuals to upskill on the latest technologies.