Hello and welcome to this lecture where I’ll be discussing AWS Glue Studio, which is one of the tools available in the AWS Glue ecosystem. AWS Glue Studio is where you create, submit and monitor your ETL jobs. 

With AWS Glue Studio, every ETL job consists of at least three things: 

1. A data source. This could be the Data Catalog, or a service like Amazon S3, Amazon Kinesis, Redshift, RDS, DynamoDB or another JDBC source. 

2. Then, you need a transformation script. Glue will use the data from your source and process it according to the transformation script you write. You can write these in either Python or Scala. 

3. Last, you need a target. Glue will export the output to a target of your choice, such as the Data Catalog, Amazon S3, Redshift or a JDBC source. 

Let’s look at Glue Studio in the Console. Here I am in the Job dashboard of the service. If I want to create a job, you can see there are many options to do so. However, they are categorized in one of two ways: I can either create a job programmatically or I can use a visual interface. 

For example, if I click the visual with a blank canvas option and click create. I can then create graphical relationships between a source, transformation scripts, and a target destination.

Let’s build one quickly. I can use the Data Catalog as my source. For my transformation script, I’ll use a built-in script called Rename Field, that renames a key in my data set to another name. Then, I can output the transformation to an Amazon S3 bucket. I can additionally choose to update my Data Catalog or not. 

While this is a pretty simple ETL job, you can create more complex relationships and graphs between services without coding at all, and Glue will generate the Apache Spark code for you behind the scenes. Note, that if you want a true no-code tool for creating ETL jobs, this won’t really provide you with that, as the built-in transformation scripts in Glue Studio are very limited. You only have about 10 options or so here. If you feel comfortable with coding, you can create custom transformation scripts in this interface using Python or Scala as well.

However, there are better places where you can develop your own custom scripts. For example, if I click back, you can see the other options for programmatically creating scripts, such as the Spark script editor, the Python shell script editor, or the built-in Jupyter Notebook interface to create Python or Scala job scripts.

That’s it for this one - see you next time.