Getting Data In
Getting Data Out
BigQuery is Google’s managed data warehouse in the cloud. BigQuery is incredibly fast. It can scan billions of rows in seconds. It’s also surprisingly inexpensive and easy to use. Querying terabytes of data costs only pennies and you only pay for what you use since there are no up-front costs.
This is a hands-on course where you can follow along with the demos using your own Google Cloud account or a trial account. You do not need any prior knowledge of Google Cloud Platform and the only prerequisite is having some experience with databases.
- Load data into BigQuery using files or by streaming one record at a time
- Run a query using standard SQL and save your results to a table
- Export data from BigQuery using Google Cloud Storage
- Anyone who is interested in analyzing data on Google Cloud Platform
- Experience with databases
- Familiarity with writing queries using SQL is recommended
- A Google Cloud Platform account is recommended (sign up for a free trial at https://cloud.google.com/free/ if you don’t have an account)
The GitHub repository for this course is at https://github.com/cloudacademy/bigquery-intro.
Welcome to “Introduction to Google BigQuery”. My name’s Guy Hummel, and I’m a Google Certified Professional Cloud Architect and Data Engineer. If you have any questions, feel free to connect with me on LinkedIn and send me a message, or send an email to firstname.lastname@example.org.
This course is intended for anyone who’s interested in analyzing data on Google Cloud Platform.
To get the most from this course, it would be helpful to have some experience with databases. It would also be helpful to have some familiarity with writing queries using SQL, but it’s not a requirement.
This is a hands-on course with lots of demonstrations. The best way to learn is by doing, so I recommend that you try performing these tasks yourself on your own Google Cloud account. If you don’t have one, then you can sign up for a free trial.
To save you the trouble of typing in the URLs and commands shown in this course, I’ve created a GitHub repository with a readme file that contains all of them. The link to the repository is at the bottom of the course overview below.
We’ll start by running some basic queries and saving the results.
After that, I’ll show you how to load data into BigQuery from files and from other Google services.
Then, you’ll see how to stream data into BigQuery one record at a time.
Finally, we’ll wrap up with how to export data from BigQuery.
But first, I’ll give you a quick overview of why you’d want to use BigQuery.
Data warehouses have been around for decades. When databases first became popular, they were primarily used for transaction processing (and that’s still the case today). But managers also needed to analyze data and create reports, which is difficult to do when the data resides in numerous databases across an organization. So data warehouses were created to collect data from a wide variety of sources, and they were designed specifically for reporting and data analysis.
If data warehouse technology has been around for so long, why did Google release BigQuery, and why would you use it instead of a more established data warehouse solution? Well, for two main reasons: ease of implementation and speed.
First, building your own data warehouse can be expensive, time consuming, and difficult to scale. With BigQuery, on the other hand, the only thing you have to do to get started is load your data into it. And you only pay for what you use, so you don’t need to spend a lot of money building capacity to handle peak periods.
Second, even if you do build your own high performance data warehouse, it will probably never be as fast as BigQuery because BigQuery can process billions of rows in seconds. This speed is especially valuable if you need to perform real-time analysis of streaming data, such as from online gaming systems or Internet of Things sensors.
Okay, now if you’re ready to learn how to crunch big data with ease, then let’s get started. We’d love to get your feedback on this course, so please give it a rating when you’re finished.
Guy launched his first training website in 1995 and he's been helping people learn IT technologies ever since. He has been a sysadmin, instructor, sales engineer, IT manager, and entrepreneur. In his most recent venture, he founded and led a cloud-based training infrastructure company that provided virtual labs for some of the largest software vendors in the world. Guy’s passion is making complex technology easy to understand. His activities outside of work have included riding an elephant and skydiving (although not at the same time).