While we build scalable, highly available and fault tolerant systems on Amazon Web Services, it is important to look out AWS deployment tools which can handle system and application level deployment ensuring consistency, predictability and integrity across multiple environments. This leads to continuous and faster deployment, lower failure and error rates and quick recovery.
There are multiple deployment tools available across the web. We have picked up the 5 best AWS deployment tools which has solid integration with the Amazon cloud or are part of AWS ecosystem.
The 5 Best Tools for AWS Deployment
Chef is one of the most popular configuration management and deployment tool widely used across enterprises. It was launched in 2009, developed in Ruby and licensed under the Apache open source license. Chef is available in 3 versions: hosted chef (SaaS solution); private chef (enterprise Chef behind firewall) and the open source version.
The Chef Infrastructure comprises of 3 components:
• Master Server: heart of the environment which holds all the information about the environment, like cookbooks and recipes (configuration units to install, configure packages, manage applications and deploy apps), nodes roles and policies, metadata about nodes etc. The Chef server supports all the most important Linux versions.
• Workstation: these are the nodes where cookbooks are developed and synchronized with Chef Master Server’s repository using Knife (a tool to manage cookbooks, nodes, roles etc.). Workstation is supported on Linux, Windows and Mac OS X.
• Client Nodes: nodes are the systems which are managed by chef-client. Chef-client is an agent which continuously run on the nodes and interacts with chef-server using its own combination of public-private key pairs. Chef-clients fetches the instructions from chef-server and execute them on that node. Chef clients can be installed on all the major operating systems thereby ensuring that one can manage any system and application using Chef.
To develop your own cookbooks, one needs to have some understanding of Ruby. Chef is heavily used by large organizations like Facebook, Mozilla, Airbnb etc. Actually, AWS Opsworks internally relies on chef recipes to install and manage packages, manage services and deploying apps.
Today, most of the major cloud computing players provide a easy-to-use UI to build your IT infrastructure on the cloud. However, unlike provisioning once on on-premise infrastructure, you may have to dynamically provision (or deprovision) dozens of virtual machine (VM) instances, a few instances of dynamic storage and some SaaS-based services. In addition, software releases need to be pushed regularly (weekly, daily or even hourly in some cases).
One way to go about it, is to create VM images for every change and create a new VM instance to push it. However, this is laborious and prone to errors especially if different instances have different application data. Add to that, What about the storage? Databases? Network configuration? What about the architecture? As your usage of cloud infrastructure for Dev/QA/Production environments grow, it becomes an operational challenge to manage the entire infrastructure. Operational tasks listed below becomes a nightmare for a System admin.
• Creating instances
• Configuring instances with storage, services, firewall, software
• Monitoring and deleting instances
• Ensuring all instances in a layer (web/app) are in same state.
This is when you would need a configuration management system which basically gives you the ability to deploy, update, and repair your entire application infrastructure using nothing but pre-defined, automated procedures. Ideally, you want to automatically provision your entire environment from bare-metal all the way up to running business services completely from a pre-defined specification, including the network configuration.
Chef is an infrastructure automation framework that makes it easy to setup, configure, deploy, and manage servers and applications to any environment (physical/virtual/cloud).
With Chef you can code your infrastructure (called ‘recipes’) and use the recipes to setup the infrastructure.
Once automated, you hold a blueprint for your infrastructure, enabling you to build (or rebuild) automatically in minutes or hours – not weeks or months. Better still, in case of disasters (network, hardware or geographical) Chef makes disaster recovery job easier.
Chef has become one of the widely used tools for configuration management. Apart from Chef, tools supporting cloud environments are Puppet, Ansible, Salt. AWS OpsWorks is an application management service that makes it easy for DevOps to model and manage the entire application from load balancers to databases. Amazon OpsWorks supports Chef.
With Chef you will be able to:
• Manage servers by writing recipes.
• Integrate tightly with applications, databases and more.
• Configure applications that require knowledge about your entire infrastructure
• Create perfect clones of QA environments, pre-production environments, partner preview environments and more.
Before we get started working with Chef, let us run through the terms used frequently.
|recipe||Configuration element within an organization. Recipes are used to install, configure
software and deploy applications
|cookbook||Fundamental unit of configuration and policy distribution. Each cookbook defines a scenario, such as everything needed to install and configure MySQL.|
|knife||Knife is a command-line tool that provides an interface between a local chef-repo and the Chef server. Knife helps provisioning resources, manage recipes/cookbooks, nodes & more.|
|chef-repo||Chef-repo is located on the workstation and contains cookbooks, recipes, roles. Knife is used to upload data to the chef server from the chef-repo.|
|workstation||A workstation is a computer that is configured to run Knife, to synchronize with the chef-repo, and interact with a single server. The workstation is the location from which most users will do most of their work.|
|node||A node is any physical, virtual, or cloud machine that is configured to be maintained by a chef-client|
|run_list||A run_list is an ordered list of roles and/or recipes that are run in an exact order.|
|chef-client||A chef-client is an agent that runs locally on every node.|
Chef comprises three main elements: a server, one (or more) nodes, and at least one workstation.
• The server acts as a hub that is available to every node. All chef client nodes will be registered with the server. The server holds all the cookbooks, recipes and policies. Clients communicate with the server to get the right configuration elements from the server and apply it to the nodes.
• The workstation is the development machine from which configuration elements likecookbooks, recipes and policies are defined. Configuration elements are synchronized with the chef-repo and uploaded to the server with knife command.
• Nodes contain chef-client which performs all the infrastructure automation.
There are 3 types of Chef Servers
1. Hosted Chef: Hosted Enterprise Chef is a version of the Chef server that is hosted by Chef. Hosted Enterprise Chef is cloud-based, scalable, and available service with resource-based access control. Makes life easier, you will not have to run additional server and manage it.
2. Enterprise Chef: is similar to hosted chef but the chef server will be located on premise.
3. Open Source Chef: is a free version of Chef Server.
In the next post, we will get started with Open source Chef on Amazon Web Services.
Along with Chef, Puppet is another deployment and configuration management tool widely used in organizations of all sizes. An initial version from PuppetLabs was first released in year 2005. It was initially launched as Free Software under the GPL license until version 2.7, but later they switched to Apache 2.0.
Puppet comes into two variants: Puppet Enterprise (free up to 10 nodes) and Puppet Open Source (completely free). Puppet is also written in Ruby.
On an abstraction level, Puppet is quite similar to Chef. It also works on a server-client model where one has to install Puppet agents on managed nodes, and centralized administration happens on the Puppet Master/Server. Puppet agents contact Puppet Master/Server periodically (say 15 minutes) and fetch the latest configuration. Once fetched, this configuration is executed on Puppet clients and results are sent back to Puppet Master. To configure your Puppet clients with relevant resources and to a state, Puppet Modules are used. These modules are either written in the Puppet-specific language based on Ruby, or in Ruby itself, and then stored on Puppet Master/Server. Each Puppet Module has its own purpose, like configuring NTP, MySQL or Tomcat, etc.
Puppet Master is only supported on Linux distributions, while Puppet clients can be run on Linux, Windows and Mac OS X.
Released in 2012, Ansible is the one of the youngest and fastest growing open source deployment, configuration management and orchestration tool. Unlike Chef and Puppet, Ansible relies on an agentless architecture, that is: it does not require any client package installation on client nodes apart from regular Python packages. The management of client nodes happens over SSH protocol. Ansible’s agentless architecture makes the upgrade process simple and easy to implement.
Ansible is availalbe in two versions: Ansible tower (the paid version) and Ansible open source (the free one).Ansible is written in Python and is licensed under the General Public License (GPL). One of the advantages of using Ansible is that is uses YAML syntax for its configuration files, also known as playbooks. A very nice choice, given that YAML is quite easy and avoids the unneeded complexity of major languages.
There are two types of nodes: Control Machine and Managed Nodes. The Control Machine is the one where Ansible is actually installed. It supports most of the Linux distributions and requires Python 2.6+ installed. Managed nodes requires Python 2.4+ and supports Linux, Windows and Mac OS X.
AWS Elastic Beanstalk
If you are looking for the fastest, simplest and maintenance free way to deploy your application on AWS, AWS Elastic Beanstalk is definitely up for consideration. AWS Beanstalk is a free service provided by AWS where you only have to pay for resources provisioned by the Beanstalk environment
AWS Beanstalk allows to deploy applications written in many different languages like PHP, .NET, Ruby, Java, Node.js, Python, and other, and has native Docker support for various web and applications servers like Apache, Tomcat, IIS, Nginx etc. Its features include:
• Quick Deployment: Uploading application files to Beanstalk initiates the deployment process on your EC2 instances. In case of a failure, one can rollback to the previous version.
• Integration with other AWS services like Autoscaling, Elastic Load Balancer, SNS, CloudWatch, RDS, etc.
• Application health monitoring using CloudWatch and SNS notifications in case of any issues.
• Easy access to application and system logs, even without logging into instances.
• Customized software and applications by passing configuration files to the AWS Beanstalk environment. These configurations files are written in YAML or JSON formats.
As it is a PAAS managed service by AWS, it frees up the organization from the burden of deployment and configuration management.
AWS Code Deploy
If one is looking for a simple code deployment service, he/she should definitely look into AWS CodeDeploy, The new service launched a few weeks ago during the AWS re:Invent 2014 in Las Vegas. AWS CodeDeploy provides several features which definitely simplify the deployment process:
• Minimize downtime: tracks the application health and performs rolling updates across deployment targets. You can deploy the previous revision in case of any failure.
• Automatic Deployment: enables deployment across different environments and thousands of deployment targets.
• Integration with existing 3rd party tools: works with existing configuration management tools (like Chef, Puppet, Ansible), version control tools (GitHub, AWSCodeCommit etc.) and continuous integration tools (Bamboo, Jenkins, CircleCI etc.)
• Centralized Management: you can execute and monitor the deployment process CodeDeploy also provides reporting feature for your deployment process.
• Integration with other AWS services: it works with AWS CloudFormation, AWS OpsWorks, AWS Beanstalk, AutoScaling, etc.
Apart from code deployment, AWS CodeDeploy also enables running scripts and setting up permissions during multiple lifecycle events like ApplicationStop, BeforeInstall, Install,AfterInstall, ApplicationStart etc. These lifecycle events are written in YAML formatted AppSpec (Application Specific) files, similarly to Ansible.
To get advantage of it, you need to install CodeDeploy agents on your Linux and Windows instances. Tested agents are available for Amazon Linux, Ubuntu and Windows. For other operating systems, open source versions of CodeDeploy agent are available. Currently, it is only available in N.Virginia and Oregon region with no additional charges.