- Here we go.

- Okay, so Packer is a way for automating the way that you build an image. So it's pretty simple, but it's really powerful, if you use it with certain workflows, and I'll demo that today. But Packer's actually kind of a competitor to configuration management tools like Chef, Puppet, and Ansible. HashiCorp has their own vision on how infrastructure should be. And Packer kind of compliments that, and that's the immutable infrastructure. You probably hear that term is kind of a buzzword, but essentially this top diagram is the traditional way that most companies deploy changes to their environment. They have their server image, they provision their image into an instance, and then any change they make, they apply it on top of each change with Puppet or Chef or Ansible. And some of the downsides for this is you have to have really good code coverage with your configuration management tool. So like for your windows server, you're not going to configure each and every setting in Puppet and enforce all those settings. You're just going to have the ones that are important to your company. The other way of provisioning infrastructure is the immutable way, which Packer really compliments. So here you have your server image and then you provision your instance and any change you need to make you go back and you change the image and redeploy that server into the environment. So you don't have to have a hundred percent code coverage on your change because any sort of change you make is like rebuilding the server again. Just kind of like how containers work. It's like the exact same way. So that's kind of HashiCorp's vision of how server infrastructure should be. It should fall more in the lines of like your containers with your Dockerfile. And Packer really allows you to do that. And one of the cons to having this sort of method with your server instances is your application architecture has to support it. So there's not really a clean way of doing this, going back on your old software and then kind of revamping it into this architecture. You kind of have to stick with the traditional method, but a lot of people are starting to build, if they're building new software, they're starting to follow this architecture. And actually a survey came out, wonderful, off Stack Overflow for 2020, and Chef and Puppet are now some of the most dreaded tools in DevOps of this year. So a lot of people are getting off of Chef and Puppet and they're going towards more of the immutable tools like Chef, oh not Chef, like Ansible and Terraform. So that's pretty interesting.

- You don't see Packer at the top of the most loved list, do you? That would be perfect.

- So yeah, a lot of people don't align Packer with like a DevOps tool. It's kind of been hidden for a while, but it's starting to get more presence now. It's actually starting to grow quite a bit, but yeah, I assume eventually it'll make it on the list, but yeah, Terraform's on there, I think it's the most loved, one of the most loved. So I would draw Packer and Terraform because they both work really well together and I'll show it in the demo.

- I guess one thing for the lab's perspective is that startup performance compared to configuration management would be much better with Packer.

- Yes, yep. A hundred percent. Because you're doing all the configuring in the image as opposed to after it's deployed. So, yep. So Packer it's just the binary and you feed it a JSON file. And that JSON file has all the configuration for building the image. And it's really simple. You have your variables, your builders, and provisioners. And your variables are the configuration settings that you wanna like feed into your configuration. So up here AWS access and secret key, I'm feeding that into my Packer configuration. So it can use that here. And the builder section is essentially, it's just your configuration for the environment that you want to build an image in. So here we have an AWS environment and I have all the settings in order to build an AMI and AWS. And down here, we have our provisioners. Your provisioners are the configuration that you're applying to that image. So essentially what Packer's gonna do is it will stand up in EC2 and AWS and it will SSH into it and it will run these commands to provision it. And then it will de-provision that EC2 instance and then make it into an AMI. So really simple. And it just does all the heavy lifting to you. So you don't have to script it out yourself for each environment. So we'll go ahead and show, so Packer validate is the command to just like does static analysis on your configuration, you just do, packer validate. I spelt it wrong. And then if there's an issue it will actually air out. So that works really well for pipelines and I'll get it. So yeah, it's really good to add that as a step into pipelines. And then to do the build, it's just packer build, and then JSON file, and now it'll reach out into AWS, deploy an EC2 instance, and then it will run these commands to install redis basically, and then turn it into an AMI. One thing to note in this builder section, and this is really interesting. So I can have multiple builders in here. I could have multiple AWS environments and I can push out an image into all those environments in parallel. So here's a really good example on their website. They're pushing out an image to AWS and digitalocean, and then it's the same provisioner that I'm using to install Redis on. And it'll just do that in parallel when you're on your packer build. So you can manage one image in multiple environments with the same code, which is really, really powerful when you start getting into multi-cloud.

- And do you know if one of the builders is like a Docker image?

- Yeah, I was gonna show this. So builders, their documentation is really good. So these are all the builders that they have. There's a lot. This has grown a lot in the last year. Oracle, Hyper-V, VMware, I've used the VMware one, Docker, so yeah, they keep adding ones, and then you can create your own. There's people that have been doing that. So yeah, they have a lot of different environments that you can build stuff in, for sure.

- Does it always depends on SSH or would it work for

- No. So there's different provisioners that you can use. Right now I'm just using the shell provisioner, to run shell commands through SSH. You can also use Chef. So all you can take all your old configuration management scripts and then use those to build your images as well, to reuse all that stuff. So Puppet, Ansible, Salts, Inspec, so you can transfer files to it. Yeah, there's a lot of different things that you can do to build out your image, however, you need. So yeah, this is wrapping up here, it's stopping the instance, creating the AMI, and then that's it. We have our AMI and AWS. Now, if we wanted to take this a step further, we can create a CI/CD pipeline with Terraform. So we could have Packer build the image and then have Terraform deploy the server instance. And then as far as like routine management, or if you wanted to add an additional package to the server image, and then she'd go and make your change, and then your CI/CD pipeline would go ahead and build it and then provision it out automatically. And we'll go ahead and show that real quick. I have a repository in Azure DevOps here, and I have my Terraform code for a server. I have a web server that's sitting in AWS that I deployed with my Terraform code using an AMI that was built by Packer. So let's say we wanted to go ahead and add a new package in here, let me just make a new branch, add snapd. Then we just modify our Packer configuration. Also, you can use a script. So I could have a script file within this images folder. And just call that with the packet duration. That's kind of the way that I've always done it. So then I don't have to muddy up this configuration. I can make it clean and just manage my script for provisioning. So change this to snapd. Change to this branch, and you will do a pull request. And this is a remote repository that it's in as your DevOps. Oh, my branch is in Azure DevOps now, and this is just like your standard workflow with create pull request. This my new change, lemme just look through it. Okay. So now this pull request will trigger a pipeline build to start building out this image, and then it will redeploy the server with the new AMI image. So Terraform will look and see that that server exists and see that the AMI, that that server used is different than the new one and it will destroy it, and then reprovision it.

- How does it see that it's different, like what's the connection?

- I'll show you that in a sec. So here's my pipeline. I broke it up into two different stages, the build and then the deploy. So the way that Packer passes that new AMI over to Terraform. So Terraform is configured right now with a variable for AMI ID. And that's passed into the integration for the EC2 instance, and here on the packer configuration, the bottom here, I've added this post processors. So post processors with Packer is anything that you wanna do, like after the image is built. So here, all I'm doing is I'm using the, it's called the manifest post processor. And that creates a manifest JSON file with any sort of attributes of the image that it created. So it actually creates the AMI ID, and here in my, let me show you here. So here I'm grabbing that manifest JSON file, before I run Terraform, and I'm saving that into a variable and then just passing it straight in to the Terraform configuration through here. So I'm just updating the AMI ID with the new one. So that's how you would pass that through. Also, host processors are pretty interesting. You can do a lot of different things with them. Here's all the posts, no, here's all the post processors that you can do. Docker save, push, you can tag things after the image is built. This Amazon import one is really interesting. So here's an example they have where they built an image on VMware, and then they use a post processor to actually push that up into an S3 bucket. So you can do a lot of additional, crazy things with post processors and Packer. So that's kind of how that works right now. It's still building.

- So the Packer itself doesn't have a state like Terraform. You can't do packer output AMI ID.

- Nope. Nope. Yeah. You have to use files. You have to use a little duct tape and glue to do it, but yeah, it works pretty nice that you can output that stuff into that manifest file. Then you just parse that through and feed it into your Terraform configuration. So yeah this finishes, it will just rebuild this EC2 instance with the snap that, or snapd package. So yeah, it's pretty slick. Cause all I had to do is just modify the image file and then it just handles it. So you can get really fancy and do like zero downtime deployment roll outs with EC2 clusters and stuff. So yeah, really powerful compared to the traditional way of just trying to make changes on the fly and roll them out. That gets really complex with Puppet and Chef. I've had to do that before. It's a lot easier to just change the image and then just roll it out. So that's why that's becoming a little more popular now.

- And just for completeness, there's nothing that says you have to use Terraform. You could have used CloudFormation or something else.

- Oh yeah. To push out the AMI. All I'm doing is just passing it through into Terraform through a variable. So yeah, if you can do the same with CloudFormation, you would just replace the stuff with the CloudFormation commands and yep, same thing. So are there any questions at all?

- Well, thanks for the nice multi-cloud tech talk there using Azure and AWS.

- But I'll let Stefano and Luca ask first if they have any questions.

- Okay.

- Nothing really important from my side. I'm wondering how we can integrate it or use it. I think or I know Logan from Stock Pro, where he was an image builder for AWS, to be a delight there. And probably will do the same for EKS, but maybe Packer could be a good alternative.

- Yeah it really shines when it's multiple environments. So if you have to maintain one image all over the place. At my previous company, we had a standardized image that was like approved by security. You had to go through all these security gates to verify this image. And so in order to maintain it in all the environments that we had AWS and Azure and in our VMware environment, we just used Packer to push it out. And it was really, really nice. So we didn't have to manually maintain it in each environment. So yeah to me, that's where it's really powerful.

- Yeah, I thought it was interesting that you can use Packer to make containers because usually, you'll think if you can make a container, we might prefer to do that versus a virtual machine image, but Packer can sort of handle both cases for when you can't actually use containers. And I think our EKS is a good example that we can't just run EKS using a container, it means we have to have the AMI.

- You can also use TerraTest to do testing with Packer and like deploying a container. It's pretty, pretty interesting. So yeah, you can test your container builds.

- Oh, Stefano.

- Well, what about performance? So is it faster than the folder typically managers such as CloudFormation, or a cloud deployer manager for GCP? Did you find a comparison between them?

- I don't expect it to be much slower because it's just manipulating the APIs for each environment. I know for Azure, if you do it through the image builder, you're using the same API, so you're not going to see any difference there. Another thing to note is, if wherever I'm doing my packer build, like here I'm SSHing into the VM. So there's network constraints that you have to be considerate about. And if you're doing any sort of file transfers from your packer repository to that server, you gotta account for that too. So I guess, yeah, it depends really what you're doing.

- Thanks. And what about the usage with the programming languages? Does Packer provide you a kind of SDK to be used with Python, Node, or with other programming languages? Or it only provides you the package shell, power shell and the,

- So you can make your own custom extension. It's all written in go though. There's no other languages. So they have a way to make your own little plugins, for like, if someone's developing some application, and they wanna have their users to be able to use Packer with it, they can make your own builders for it. As far as like other languages, like, you know how Terraform has the Python SDK, they don't really have anything like that, that actually manipulates the packer building. They're converting right now, Packer from JSON. So these configurations on JSON, they're configuring it to HCL, which is the language that Terraform configurations are written in. So once they get that done, I assume they'll start using like the Python SDK that they're using for Terraform with Packer at some point.

- Okay thanks.

- Yeah.

- And just to wrap up, Luke, any features you would like to see Packer implement, or is there anything that you think is missing from Packer yet?

- So when I first started using it, yeah, there was a lot of like pain points, especially on the VMware side. You could only build VMware images on like a standard one server. It was kind of a pain in the butt, but over the last year, like this product has just like exploded. So as far, like right now, I don't really have any issues or pain points using Packer. It's gotten so much better just in the last like six months. And I assume it's gonna get even better and better because they have more people on the team. I think six months ago, there was only two people working on Packer. It was like that small. And since then, they've just expanded that team out at HashiCorp. So we'll probably see a lot of cool things come out of this.

- Is it like Terraform that most of these builders are hosted on GitHub so you can make pull requests if there was something missing?

- Yeah. It's all open source.

- Okay.

- Yeah.

- All right. Any last questions guys or comments? Okay Luke, thanks for the great presentation and going the extra mile with that multi-cloud demo.

- Yeah.

- Okay. Take care guys. Luca, hopefully, it was useful, maybe. I don't know if it can outperform image builder, if you guys are AWS native for everything, but I think it's worth considering.

- Yeah. Yeah. Thanks. Excellent.

- Yep.