Getting Data In
Getting Data Out
The course is part of these learning paths
BigQuery is Google’s managed data warehouse in the cloud. BigQuery is incredibly fast. It can scan billions of rows in seconds. It’s also surprisingly inexpensive and easy to use. Querying terabytes of data costs only pennies and you only pay for what you use since there are no up-front costs.
This is a hands-on course where you can follow along with the demos using your own Google Cloud account or a trial account. You do not need any prior knowledge of Google Cloud Platform and the only pre-requisite is having some experience with databases.
- Load data into BigQuery using files or by streaming one record at a time
- Run a query using standard SQL and save your results to a table
- Export data from BigQuery using Google Cloud Storage
There are two components to BigQuery pricing. Storage and queries.
BigQuery storage charges are incredibly cheap. It costs two cents per gigabyte per month, which is the same price as cloud storage regional. What's even better is that if you don't edit a table for 90 days, then the price for that table drops to one center per gig per month until you modify the data in the table again. That's as cheap as near-line storage. In fact, it's even cheaper, because when you read data from near-line storage, there is a one-cent-per-gig charge. With BigQuery storage, you aren't charged for reading data at all.
Since we've only been using public datasets so far, there won't be any storage charges.
The only other charge is for queries. There's also a charge for streaming data to BigQuery in realtime, but that doesn't apply to these examples and I'll cover it in another lesson. For queries, the first terabyte per month is free. After that it costs five dollars per terabyte, which is half a center per gigabyte. Wait a minute, didn't I just say that you aren't charged for reading data from BigQuery storage? Yes, that's true, because BigQuery charges query fees regardless of where you read the data from. For instance, if you query a dataset that's in cloud storage, then you get charged at the same rate that you would from querying a dataset in BigQuery storage. So the charge isn't for reading, it's for processing.
For high-volume customers there's also flat-rate pricing. But it's only worthwhile if you spend at least $10 thousand per month. It only applies to query costs and not storage, which is still separate.
To see how much data is processed by a query, look in the validator message area. If it's not showing, click the validator circle again. Since there isn't an error in the syntax, now it's showing how much data would be processed by the query above. In this case it's 163 meg. Considering that the first terabyte of processing in a month is free, this won't cost us anything. But even if we were already over the one terabyte mark this month, it wouldn't cost much. How much? Less than a tenth of a cent. I'd say that's pretty reasonable.
And that's it for this lesson.
About the Author
Guy launched his first training website in 1995 and he's been helping people learn IT technologies ever since. He has been a sysadmin, instructor, sales engineer, IT manager, and entrepreneur. In his most recent venture, he founded and led a cloud-based training infrastructure company that provided virtual labs for some of the largest software vendors in the world. Guy’s passion is making complex technology easy to understand. His activities outside of work have included riding an elephant and skydiving (although not at the same time).