If you are working with models that handle recorded event data or facts, you could fall into a situation where your models are completely refreshed at every dbt execution and most of the data was already in place in your tables. This may be because these types of data rarely change once created (e.g bank transactions).
Incremental dbt models work well in these scenarios and help speed up the dbt model creation and not overhead databases. Incremental models are materialized as tables, but they are not fully refreshed at every dbt execution. Only new records that were not previously in the table are added.
In this lab step, you will create two dbt incremental models by using two different strategies in order to let dbt understand whether a row needs to be added or not.
Learning Objectives
Upon completion of this lab, you will be able to:
- Understand what are incremental models
- Work and create incremental dbt models
Intended Audience
- Data engineers with a focus on data modeling and transformations methodologies
- Developers that need to understand different kinds of dbt materialization techniques
Prerequisites
To get the most from this lab, you should have basic knowledge of dbt. To get ready, you can use the following labs:
Stefano studies Computer Science and is passionate about technology. He loves working with Cloud services and learning all the best practices for them. Google Cloud Platform and Amazon Web Services are the cloud providers he prefers. He is a Google Cloud Certified Associate Cloud Engineer. Node.js is the programming language he always uses to code. When he's not involved in studying or working, Stefano loves riding his motorbike and exploring new places.