Boto: Using Python to Automate AWS Services

Boto allows you to write scripts to automate things like starting AWS EC2 instances

Boto is a Python package that provides programmatic connectivity to Amazon Web Services (AWS).

AWS offers a range of services for dynamically scaling servers including the core compute service, Elastic Compute Cloud (EC2), along with various storage offerings, load balancers, and DNS. You can control these services either through the AWS console or by way of AWS’s extensive API. But there are a number of third-party libraries for using this API. For Python, we have Boto which allows you to write some handy scripts to automate simple things like starting or stopping EC2 instances or taking regular snapshots of your servers.

Depending on your Python experience level, you may want to get some basics down or brush up on some more advanced skills. Whatever level you’re at, we offer a thoughtful series of courses to help you educate yourself at your own time and pace. New programmers can jump right into Python for Beginners Learning Path. Programmers with some experience can tighten up their coding chops with the Pythonic Programming Learning Path. With Cloud Academy, you can train with scenario-based Learning Paths and thousands of Courses, Quizzes, Exams and Hands-on Labs.Check out the link below to our Python courses, and find unlocked courses you can take right away.

Cloud Academy Python Training

Installing Boto on Linux (CentOS)

1. You’ll first use the yum package manager to install Python and the pip Python package installer:

yum install python python-devel python-pip

2. Now we will install the Boto package via pip:

python-pip install boto

3. Finally, we can test if everything has been successfully installed on our Linux box:

python -c "import boto; print boto.Version"
2.9.6

Configuring AWS credentials

On startup, the Boto library looks for configuration files in the following locations (in this order):

  • /etc/boto.cfg – for site-wide settings for all users on this machine
  • ~/.boto – for user-specific settings

Use cases

Lets start with a basic scenario:  launching an EC2 instance

#/usr/bin/python
import boto.ec2
conn = boto.ec2.connect_to_region("us-west-2")
conn.run_instances(
    'ami-6ac2a85a',
    key_name='nitheesh_oregon',
    instance_type='t1.micro',
    security_groups=['nitheesh_oregon']
)

The above python script will connect to Amazon’s us-west region and launch an instance using AMI id: ami-6ac2a85a.

Scheduling stop instances. Let’s say you now want to stop your test environment instances at a certain time each day. Run:

#/usr/bin/python
import boto.ec2
conn = boto.ec2.connect_to_region("us-west-2")
conn.stop_instances(instance_ids=['instance-id-1','instance-id-2'])

You can schedule this script to run every day at a particular time by creating a cron job.

CloudFront invalidation. AWS doesn’t provide a command line tool for CloudFront. Therefore, if you want to invalidate objects, you’ll need to log into the console and add the objects to invalidate. It’s a really long and tiresome process. Instead, you can make use of the Boto library:

#/usr/bin/python
import boto
dist_id = 'xxxxxxxxxxx'
invalidation_path = '/home/ec2-user/invalid.txt'
def main():
    paths = open(invalidation_path,"r+")
    conn = boto.connect_cloudfront()
    inval_req = conn.create_invalidation_request(dist_id, paths)
    print inval_req
    touch = open(invalidation_path,"w")
    touch.write("")
if __name__ == '__main__':
    main()

This script reads objects specified in a file, invalidates them, prints an invalidation ID, and clears the contents of invalid.txt.

AWS services supported by Boto

At the moment, Boto supports more than 50 Amazon services, running the whole range from compute, database, application, and payments and billing. You can find a complete and current list on the Python.org website.

Updates to Boto

As a note, Boto3 is the latest version of Boto, which is considered to be the Amazon Software Developers Kit (SDK) for Python. One of the main ways in which Boto3 differs from the original Boto in that the newest version is not hand-coded, and therefore is is kept continually up-to-date for the benefit of its users. You also get the benefit of the most modern API types, and full native support for Python versions 2 and 3.

Conclusion 

Boto lets you write scripts that manage complex setups in AWS. In addition, it provides support for other public services such as Google Storage, besides private cloud systems like Eucalyptus, OpenStack, and Open Nebula.

If you’re a Python expert and value your time, go for it. If you’re new to Python, Cloud Academy offers an Introduction to Python Learning Path that guides you through the background and basics of Python so you get you the skills and knowledge you need to get started quickly as possible.

Cloud Academy Introduction to Python

For a different way of using Python on AWS, see our blog on Amazon Machine Learning.

Cloud Academy