Organizing and responding to events in a decentralized system (such as with microservices or IoT) can be a challenge. In this course, you will learn how to use Google Cloud Pub/Sub to create a reliable, asynchronous, messaging service for any scale.
Learning Objectives
- Understand what Cloud Pub/Sub is
- How to send and receive messages
- What the typical use cases are
- How to get started
Intended Audience
- GCP Developers
- GCP Data Engineers
- Anyone preparing for a Google Cloud certification (such as the Professional Data Engineer exam)
Prerequisites
- Access to a Google Cloud Platform account is recommended
In this lesson, I am going to briefly list a few use cases for Cloud Pub/Sub. Now, Pub/Sub is used for many things, including real-time event processing. You can use Cloud Pub/Sub to create a real-time, enterprise-wide, data sharing bus. Business events, user interactions, and system failures all can all be tracked and responded to as they occur.
Parallel processing. Pub/Sub can be used to efficiently distribute a large number of tasks. If you were creating a website similar to YouTube, you might wanna kick off many different jobs when a user uploads a new video. You could have one process for re-encoding the video to different resolutions. Another could be used to scan for copyright infringement. And a third process might be used for generating closed captions. All of these tasks and more can happen in parallel using auto-scaling microservices.
Tracking database changes. Pub/Sub is also commonly used to communicate database change events. Any interested services can get a notification for when a record is added, edited, or deleted. Now, these events can be used to sync multiple databases, create incremental backups, or to log all changes for later Big Data analysis.
Streaming Internet of Things data. IoT devices are constantly generating lots of data. You can use Pub/Sub to capture, store, and filter this data so that your services are not overwhelmed with lots of unnecessary notifications.
Now, these are just a few examples. Cloud Pub/Sub is also used to coordinate microservices, ingest data for streaming analytics and data integration pipelines, and much, much more.
Daniel began his career as a Software Engineer, focusing mostly on web and mobile development. After twenty years of dealing with insufficient training and fragmented documentation, he decided to use his extensive experience to help the next generation of engineers.
Daniel has spent his most recent years designing and running technical classes for both Amazon and Microsoft. Today at Cloud Academy, he is working on building out an extensive Google Cloud training library.
When he isn’t working or tinkering in his home lab, Daniel enjoys BBQing, target shooting, and watching classic movies.