What are the key traits of a data engineer? Well, data engineers in my view are practical people, right? Hands-on, they're critical thinkers, data engineers connect the dots and see the big picture to evaluate how data insights affect the business. They're collaborative. They have the ability to collaborate with both internal technology and business teams, as well as with integration partners and customers. You'd be working with a number of different people. And they're curious, so data engineers demonstrate a track record of learning and deep diving into complex systems and problems. 

Let's imagine you're working in this role. You'll be expected to design, develop, and deploy scalable data pipelines and data services. You'll be expected to promote best practices that enable teams to accelerate their consumption and understanding of data. You'll need to integrate new data development technologies and software engineering tools into your existing infrastructure. And you'll need to improve overall efficiency by introducing new engineering processes and tools.

You'll be expected to research opportunities for data acquisition and new uses for existing data. And you'll have to have an understanding of these existing systems and be able to recommend ways to marry disparate data sets using automation. You need to collaborate with data scientists and researchers to develop algorithms, including prototypes, predictive models, and proofs of concepts. So it's all very outcome orientated. As I mentioned earlier, you become a superhero because you're the person that can bring all this data to life.

Lectures

Introduction - What Does a Data Engineer Do? - Data Engineer vs. Data Scientist - Data Engineer vs. Database Administrator - Key Skills of a Data Engineer - Is This Role Right For Me?