Skip to main content

AWS Puppet Deployments

AWS Puppet for streamlined deployments

If you’re interested in automating and simplifying control over your application deployments, then Puppet – in any of its iterations – is probably a good place to start. And an AWS Puppet project can present a particularly good match.

Don’t think of this as an all-in quick start guide. The complexity of the kinds of network infrastructure that AWS Puppet deployments are designed to manage makes the word “quick” pretty much incompatible with the word “guide.” However, I believe that this post should be useful as an orientation, to introduce you to the kind of things Puppet can do and some of the ways to do them. By the way, even though Puppet is obviously not an AWS product (it’s actually the brainchild of PuppetLabs),

I’m going to use the phrase “AWS Puppet” in this post as a convenient way to describe the ways they can work together.

AWS Puppet Labs LogoHow Puppet works

Let’s say you’re running ten or fifteen Ubuntu web servers within an AWS VPC. The odds are that you will need to make configuration and content changes to each of these machines fairly often, but the process – especially as the number of servers grows – can be tedious.

Using Puppet, you can set up each server as an Agent that will, every thirty minutes or so, query a file called a manifest that lives on a separate AWS Puppet Master machine. If the manifest file, as it relates to a specific Agent, has changed since the last query, its current instructions will be immediately executed on that Agent.

This way, you can precisely and automatically control the ongoing configuration of any number of virtual (or actual) machines – running Linux/Unix or Windows – from a single file. It couldn’t get more simple. But that doesn’t mean it can’t get complicated.

Packages – AWS Puppet Master

Once you’ve created the EC2 instance that will serve as your Puppet master, you’ll have to install a Puppet package. Even if you’re limited to the open source version, there is still quite a selection of packages to choose from.

We’ll assume that we’re working with an AWS Ubuntu 14.04 AMI. For simplicity, I’ll use the puppetmaster-passenger package that’s found in the apt-get repositories. This package comes out of the box with everything you’ll need for connectivity, including Apache.

sudo apt-get update
sudo apt-get install puppetmaster-passenger

Puppet Agent

You’ll also have to install the Puppet Agent on each server – or Node – you’d like to be automatically controlled by your AWS Puppet Master. Once again, there is a choice of packages, but we’ll go with the apt-get repository:

sudo apt-get update
sudo apt-get install puppet

You’ll have to enable the Puppet Agent by editing its puppet file:

sudo nano /etc/default/puppet

…So that the START line looks like this:

START=yes

Configuration

We’re not even going to begin discussing many of the basic configuration elements you would need for a production deployment. Therefore, take it as given that you’ll have to properly configure your DNS, NTP, and SSL certificates. Here’s a great resource that addresses these issues in some detail.

You could, by the way, set up your DNS services using an old and reliable open source package like Unbound but, since we are working with AWS infrastructure, why ignore the obvious integrated solution: Route53?

You will probably also want lock the release version of each installed AWS Puppet package – both on the Master and on each Agent – to prevent unexpected version updates…which can have unpredictable results.

AWS Puppet Files

The file that matters the most for Puppet (on an Ubuntu installation) is /etc/puppet/puppet.conf. Here’s an example of what it might look like:

[main]
logdir=/var/log/puppet
vardir=/var/lib/puppet
ssldir=/var/lib/puppet/ssl
rundir=/var/run/puppet
factpath=$vardir/lib/facter
templatedir=$confdir/templates
prerun_command=/etc/puppet/etckeeper-commit-pre
postrun_command=/etc/puppet/etckeeper-commit-post
[master]
# These are needed when the puppetmaster is run by passenger
# and can safely be removed if webrick is used.
ssl_client_header = SSL_CLIENT_S_DN
ssl_client_verify_header = SSL_CLIENT_VERIFY

This config file tells Puppet (and you) where to find key system resources, including Puppet’s log files and SSL certificates.

The files that do all the heavy lifting for Puppet are called Manifests, and use a .pp extension. Here’s a simple example:

file {'/tmp/local-ip':
  ensure  => present,
  mode    => 0644,
  content => "My Public IP Address is: ${ipaddress_eth0}.\n",
}

This manifest will create a file named local-ip in the /tmp directory, assign it read-only permissions for everyone except the owner (0644), and populate it with a script that will output the public IP address of the eth0 interface from whichever machine it’s running on.

Common Puppet Commands

Start a manifest file:

puppet apply web-servers1-2.pp

Create a new group:

sudo puppet resource group puppet ensure=present

Create a new user:

sudo puppet resource user puppet ensure=present gid=puppet shell='/sbin/nologin'

Specify a single Node (or group of Nodes)

You can apply particular configuration details to specific Nodes within a general manifest file using the “node” declaration:

node 'web-server1' {
  include apache
  class { 'ntp':
    enable => false;
}
  apache::vhost { 'example.com':
    port    => '80',
    docroot => '/var/www/example',
    options => 'Indexes MultiViews'.
  }
}

(The above example also shows how to deploy pre-baked modules provided by either the Puppet team or the user community.)

Launching manifests

Finally, you can create and run manifests directly from the command line. You might, for instance, want to create a file with local IP information (using our example from above). Assuming that we named that file local-info.pp, and saved it to the AWS Puppet Manifests directory, you could run it using:

sudo puppet apply /etc/puppet/manifests/local-info.pp

If you want to learn more about Puppet, take a look at Cloud Academy’s Getting Started with Puppet hands-on course with exercises every step of the way to give you experience using Puppet.

Avatar

Written by

David Clinton

A Linux system administrator with twenty years' experience as a high school teacher, David has been around the industry long enough to have witnessed decades of technology trend predictions; most of them turning out to be dead wrong.

Related Posts

Avatar
Stuart Scott
— July 18, 2019

AWS Fundamentals: Understanding Compute, Storage, Database, Networking & Security

If you are just starting out on your journey toward mastering AWS cloud computing, then your first stop should be to understand the AWS fundamentals. This will enable you to get a solid foundation to then expand your knowledge across the entire AWS service catalog.   It can be both d...

Read more
  • AWS
  • Compute
  • Database
  • fundamentals
  • networking
  • Security
  • Storage
Avatar
Adam Hawkins
— July 17, 2019

How to Become a DevOps Engineer

The DevOps Handbook introduces DevOps as a framework for improving the process for converting a business hypothesis into a technology-enabled service that delivers value to the customer. This process is called the value stream. Accelerate finds that applying DevOps principles of flow, f...

Read more
  • AWS
  • AWS Certifications
  • DevOps
  • DevOps Foundation Certification
  • Engineer
  • Kubernetes
Avatar
Stuart Scott
— July 2, 2019

AWS Machine Learning Services

The speed at which machine learning (ML) is evolving within the cloud industry is exponentially growing, and public cloud providers such as AWS are releasing more and more services and feature updates to run in parallel with the trend and demand of this technology within organizations t...

Read more
  • Amazon Machine Learning
  • AWS
  • AWS re:Invent
  • Machine Learning
Avatar
Stuart Scott
— June 27, 2019

AWS Control Tower & VPC Traffic Mirroring

AWS re:Inforce 2019 is a two-day conference for security, identity, and compliance learning and community building. This year's keynote, presented by AWS Vice President and CIO, Stephen Schmidt, announced the general availability of AWS Control Tower and the new VPC Traffic Mirroring fe...

Read more
  • AWS
  • re:Inforce 2019
  • traffic mirroring
  • VPC
Avatar
Stuart Scott
— June 20, 2019

Working with AWS Networking & Amazon VPC

Being able to architect your own isolated segment of AWS is a simple process using VPCs; understanding how to architect its related networking components and connectivity architecture is key to making it a powerful service. Many services within Amazon Web Services (AWS) require you t...

Read more
  • AWS
  • VPC
Avatar
Stuart Scott
— June 19, 2019

AWS Compute Fundamentals Update

AWS is renowned for the rate at which it reinvents, revolutionizes, and meets customer demands and expectations through its continuous cycle of feature and service updates. With hundreds of updates a month, it can be difficult to stay on top of all the changes made available.   Here ...

Read more
  • AWS
Jeff Hyatt
Jeff Hyatt
— June 18, 2019

10 Steps for an Effective Reserved Instances Strategy

Amazon Web Services (AWS) offers three different ways to pay for EC2 Instances: On-Demand, Reserved Instances, and Spot Instances. This article will focus on effective strategies for purchasing Reserved Instances. While most of the major cloud platforms offer pre-pay and reservation dis...

Read more
  • AWS
  • EC2
Joe Nemer
Joe Nemer
— June 18, 2019

AWS Certification Practice Exam: What to Expect from Test Questions

If you’re building applications on the AWS cloud or looking to get started in cloud computing, certification is a way to build deep knowledge in key services unique to the AWS platform. AWS currently offers 11 certifications that cover major cloud roles including Solutions Architect, De...

Read more
  • AWS
  • AWS Certifications
Avatar
John Chell
— June 13, 2019

AWS Certified Solutions Architect Associate: A Study Guide

The AWS Solutions Architect - Associate Certification (or Sol Arch Associate for short) offers some clear benefits: Increases marketability to employers Provides solid credentials in a growing industry (with projected growth of as much as 70 percent in five years) Market anal...

Read more
  • AWS
  • AWS Certifications
Chris Gambino and Joe Niemiec
Chris Gambino and Joe Niemiec
— June 11, 2019

Moving Data to S3 with Apache NiFi

Moving data to the cloud is one of the cornerstones of any cloud migration. Apache NiFi is an open source tool that enables you to easily move and process data using a graphical user interface (GUI).  In this blog post, we will examine a simple way to move data to the cloud using NiFi c...

Read more
  • AWS
  • S3
Avatar
Chandan Patra
— June 11, 2019

Amazon DynamoDB: 10 Things You Should Know

Amazon DynamoDB is a managed NoSQL service with strong consistency and predictable performance that shields users from the complexities of manual setup. Whether or not you've actually used a NoSQL data store yourself, it's probably a good idea to make sure you fully understand the key ...

Read more
  • AWS
  • DynamoDB
Avatar
Andrew Larkin
— June 6, 2019

The 11 AWS Certifications: Which is Right for You and Your Team?

As companies increasingly shift workloads to the public cloud, cloud computing has moved from a nice-to-have to a core competency in the enterprise. This shift requires a new set of skills to design, deploy, and manage applications in cloud computing. As the market leader and most ma...

Read more
  • AWS
  • AWS Certifications