Hello and welcome to this lecture video. In previous lectures, we learned about the Cloud Logging Agent and how it can be used to stream log files to the GCP Cloud Logging Service. In this video, we learn to send logs programmatically using the Cloud Logging API client libraries. For demonstration purposes, I will have a sample application written in Python and will use the Cloud Logging API Python library to write directly to GCP. You can use similar methods for applications written in other languages such as Java, Go, Ruby, et cetera. The Cloud Logging API also has REST and gRPC references, but these are beyond the scope of this course.

We can break this section into three parts. First, we will install the required python library on our workstation. Then, we work on the authentication part so that our application running on the workstation can talk to GCP Cloud Logging API. Then we can jump into the Python code and begin to write log entries. So let's begin with installing the libraries.

This course assumes that you already have Python 3 installed on your workstation along with the pip utility. Here, the workstation could be your local machine or virtual machine running on your laptop or data center or VM Instance running in the Cloud. For this demo, I am going to use an Ubuntu Virtual Machine on my local machine. Here I am logged into my Ubuntu Virtual Machine and to install the GCP Cloud Logging Python library, I run the command pip install-upgrade google-cloud-logging. The google-cloud-logging package is now installed.

To verify, I run the command pip list | grep google-cloud-logging. We see that there is a package with the name google-cloud-logging and version 2.2.0. Version 2.2.0 is the latest version at the time of recording. You might see different version if this package gets an update. You can view the package release history by following the link provided in description below.

Now, we need to make sure that the VM instance running on our local machine can talk to the GCP Cloud Logging API. As this VM sits outside Google Cloud Platform, we can use a GCP Service account for authentication. To create a Service Account, let's go back to Google Cloud Platform Console UI. Here on Console UI, click on Navigation Menu and scroll down to IAM & Admin. And then select Service Accounts. On this page, you can see all the service accounts created for this GCP Project. As this is a new GCP project, I only have the default compute service account. You might see more service accounts here if you have created them for other purposes.

To create a new Service Account, I click on Create Service Account option available on top of this page. This takes us to the Create Service Account webpage, where we can fill service account details. For the Service Account name, I am putting gcp-logging-demo. You are free to choose the name based on your naming convention. After filling the name, it automatically displays the Service account ID, which is gcp-logging-demo@gcp-demo-8888.iam.gserviceaccount.com. It is also good to provide some meaningful description for the Service account.

Now, let's click on the Create button to create the service account with the provided name. Now, it provides an option to grant permission to this service account. For permissions, please follow the guidance used for your GCP workload, but generally the principle of least privilege should be followed. For this demo, I will provide the predefined editor role to this Service Account and continue. We can leave the grant users access to this service account option. This option could be useful when you have multiple users or groups in your GCP Project. Let's click on Done to finish the Service Account setup.

We have successfully created the Service account, but to use this service account outside GCP, we need its secret key. To create a key, click on the three dots displayed on the right side here. Click on Manage Keys. Here, click on the Add Key drop-down and select Create a New Key option. This will now ask the type of key we want, so select the recommended option JSON and click on Create. This will download the Service Account Key onto our local machine. As this JSON key allows access to the cloud resources, please keep it secure. Here on the Keys webpage, we can see that this Service account now has one key and it is in active status. It also shows the Key creation date and Key expiry date. So far, we have installed the google-cloud-logging python library and created a service account for authentication.

Now, let's move to the application part, where we will learn to send log entries to Google Cloud Logging Service. I am using VSCode, but feel free to use your preferred IDE such as Atom, PyCharm, et cetera. This is my sample Python-Flask application code. This application has one route, which displays a welcome message. It also handles errors for non-existence pages and displays a Page Not Found 404 message. This application code only prints the message on Standard Output and those are not stored centrally, hence it is difficult to debug any issue that occurs with the application. We will now use the google-cloud-logging library to write logs to GCP Cloud Logging Service.

First, I will import the library using from google.cloud import logging. Then instantiate a client with the name gcp_log_client. Now, we can define the log_name variable to pass the Log Name we want to use on the Cloud Logging. This line sets the logger with the log_name provided above. Now we can start writing the logs. We use the log_text utility available with logger to write the log entries. Suppose, we want to log an info level message when someone opens the root page, then we can use the log_text utility and write a message with severity as INFO. If someone tries to open a non-existence page, then we can log a message with severity as ERROR.

We are set from the code perspective, but how would this python application be able to talk to the GCP API? Remember, we created a service account. We use that service account to enable this communication. To use the GCP Service account key, we can use the standard environment variable GOOGLE_APPLICATION_CREDENTIALS. I am running the command export GOOGLE_APPLICATION_CREDENTIALS and providing the JSON key path. If you plan to run your application within GCP, for example, on a Google Compute VM Instance, then you can simply use the service account attached to the VM Instance.

Now, lets run this application using the command python3 app.py, and this opens up a local development server running on port 5000. We can now browse this page to verify the content served by the web-server. We get the expected Welcome message. Now, let's change the endpoint to some random string, let's say notfound and we get the 404 message. So our application works as expected and now let's verify that we can see the log entries in GCP. We come back to the Cloud Logging Web UI and click on Jump to Now. We can see the recently created log entries. With this, we conclude this lecture.

To summarize, we learned how to install the Cloud Logging API python client library and then how to create a Service account with a key to access GCP APIs. Then we updated our Python application code to use the google-cloud-logging package and sent log entries to GCP. In the next section, we learn to explore metrics based on the logs.