Putting it all Together
Integrating with GitHub and Google Cloud
The course is part of this learning path
This course explores how to build microservices in Go and deploy them on Kubernetes to create reusable components that can be fully managed in the cloud. We'll talk about what a microservice is and its overall architecture. We'll then take a look at the Go programming language, its benefits, and why it works well for building microservices.
You'll also get an intro to Kubernetes including what it is, what it is used for, and the key components that are needed to get a microservice from code to be exposed on the internet.
We'll then combine these three services to do an example use case where we'll build a microservice in Go and deploy it on Kubernetes. Finally, we'll look at CI/CD integration with GitHub and Google Cloud and how you can automate your deployments.
- Learn about microservices and their overall architecture
- Learn about the Go programming language and why it's good for building microservices
- Understand how Kubernetes can be used to deploy microservices
- Learn about CI/CD with GitHub and Google Cloud
This course is intended for engineers throughout the tech stack or anyone who wants to get their feet wet in DevOps and learn how programs can be managed in the cloud.
There are no essential prerequisites for this course. However, we recommend that you have:
- Experience with at least one high-level programming language, whether that be Java, Python, or Ruby
- A conceptual understanding of Linux containers and/or Docker
Now let's go through our microservice checklist from the previous section. Business requirements aside, does our code satisfy as a microservice? Do we keep things small? 35 lines of code is pretty small in my opinion. So check, is it decoupled? There are no external dependencies of any other microservices. There's also no reliance on any third-party libraries. Not that there's anything wrong with that, but again we're trying to use the tools provided to us by the standard library and not hunting for something already written that has the potential to lose maintainability. I would say yes, decoupled, check. And lastly, does it embrace the Unix philosophy? definitely.
This code is doing one thing, getting the time, and it does it very, very well. We have our code, a perfectly acceptable web application. We now need to put it inside a container. And if you're new to Docker I really recommend trying to follow along. But if you're a Docker veteran this pattern may seem pretty obvious to you.
In this example, this is a standard Docker file for Go applications. Eight lines of code, and it handles everything for us. Technically, we are going to be creating two separate containers. The first one will be our builder container. The builder container is responsible for obtaining the Go compiler and tool chain and building our microservice.
The first thing it does is pull down an already built container with the Go compiler already installed. It is then instructed to copy the source code from our host machine, into the builder container. Then it compiles the code and outputs it to the name of time-TZ, the build is complete.
Next, we want to use another container and this will be the final container that is distributed to our Kubernetes cluster. Instead of using a Go container base, we're going to use a special Docker container called scratch. A scratch container has no base, no shell, no Linux distribution.
At its core, it is a blank slate, and we're going to copy the binary that we compiled in the previous container into this empty container. The last thing needed to make this container work is to instruct it to start the web application. Building the container in this way, reduces the size of the distributed container considerably. The first container, the builder has the go run time and it is based on Debbie and Buster. It is just about a gigabyte in size, just in the build process. And that's a lot of wasted space that we don't actually need.
We have our application, a single small binary and we're going to reduce our container size by distributing a container with only our binary inside of it. Now we'll build the container using the Docker build command and given a tag of time-zoner.
To make this accessible for Kubernetes, the container needs to be published to a container repository. Since we're using Google Cloud, this will be pushed up to the Google container registry. You're free to use Docker hub or any other container registry of your choice, but it must be accessible by your Kubernetes cluster. All that's left is to serve it to the Cloud.
To serve our application, we need to build a Kubernetes cluster. You'll need to fire up your favorite web browser and go to console.cloud.google.com. From the left side menu options, find and hover over Kubernetes and select cluster. Chances are that you will not have any clusters and will be greeted by a nice button for you to create a new one. There's a lot of options to tweak in here, but for demonstration purposes, we'll be keeping the default settings.
At the time of this recording, default is three notes, which comes down to six VCPUs and 12 gigs of Ram. These of course could be reduced or expanded later as things grow. But for now, let's just keep the default settings.
If you're more comfortable on the command line, you can use the command gcloud container clusters create, and then enter the name of your cluster. Everything that we'll be doing within the Google Kubernetes engine can also be done via the command line. But for the sake of brevity, we'll be sticking with the cloud console only.
Give it a few moments for the cluster to be created. Once your cluster is deployed, the next step is to create a deployment for your application. In Google Cloud, These are called workloads. This will be on the left-hand side within your Kubernetes options.
Once you're in the workload section, up on the top right of your screen, you'll see a button labeled, Deploy. Click that, and you'll be brought to the deployment setup page. You'll need to put in the path to your container for the time zoner application we made earlier.
In my example, I'm using the Google container registry, which is within my project. So I'll put gcr.io/my projec/timezoner. Our microservice doesn't use any environment variables but if we were following the 12 factor app methodology we can add environment variables at this time and there'll be passed into the container at runtime.
Hit the continue button and set up the deployment configuration. Change the application name to time-zoner leave the default cluster and click the deploy button. Give it a minute or so to acquire the container resources and deploy your pods.
Once it is deployed, you'll be redirected to your deployment page within your Google cloud console. If the deployment is not ready yet, you can watch this progress in real time. Within your deployment page, there will be a gray bar at the top, informing you that this deployment is not yet exposed.
To expose it, we need to create a service. This notice will redirect you to the service creation page for your deployment. Once you're greeted with your service creation page, keep the service port at port 80 but make sure to define the target port of port 8,000. Port 8,000 is the port the Go Web server is listening on.
If you remember from our architecture diagram from earlier, we will be creating a load balancer service. So select it from the dropdown menu. Give it a name of time zoner service and click the exposed button. Congratulations, you have successfully deployed a microservice to the cloud on Kubernetes.
Now that your application is deployed, no task is complete without proper testing and quality assurance. Creation of the service will take a moment because Google Cloud is creating an external load balancer for you and assigning it a public IP address.
Once Google informs you that the service is up and running, find its external IP address in the table below. And you can go to that IP address in your web browser and hit the slash time end point. You will see the current time in UTC in a nice JSON format.
You can also test the TZ parameter. In the URL, append a question Mark followed by TZ and an equal sign. Try setting the parameter to Asia/Tokyo. And you will get the current time in Japan, which should be nine hours ahead of the time in UTC. Test it some more. Input and invalid time zone. Test your own. If you live in a city that happens to be two or more words say, New York or Los Angeles. Remember to replace the space with an underscore. Otherwise, your application will return an error.
Calculated Systems was founded by experts in Hadoop, Google Cloud and AWS. Calculated Systems enables code-free capture, mapping and transformation of data in the cloud based on Apache NiFi, an open source project originally developed within the NSA. Calculated Systems accelerates time to market for new innovations while maintaining data integrity. With cloud automation tools, deep industry expertise, and experience productionalizing workloads development cycles are cut down to a fraction of their normal time. The ability to quickly develop large scale data ingestion and processing decreases the risk companies face in long development cycles. Calculated Systems is one of the industry leaders in Big Data transformation and education of these complex technologies.