Skip to main content

Boto: Using Python to Automate AWS Services

Boto allows you to write scripts to automate things like starting AWS EC2 instances

Boto is a Python package that provides programmatic connectivity to Amazon Web Services (AWS).

The Amazon Cloud offers a range of services for dynamically scaling servers including the core compute service, the Elastic Compute Cloud (EC2), various storage offerings, load balancers, and DNS. You can control these services either through AWS console or by way of AWS’s extensive API. But there are a number of third-party libraries for using this API. for Python, we have Boto. Boto allows you to write some handy scripts to automate simple things like starting or stopping EC2 instances or taking regular snapshots of your servers.

Installing Boto on Linux (CentOS)

1. You’ll first use the yum package manager to install Python and the pip Python package installer:

yum install python python-devel python-pip

2. Now we will install the Boto package via pip:

python-pip install boto

3. Finally, we can test if everything has been successfully installed on our Linux box:

python -c "import boto; print boto.Version"
2.9.6

Configuring AWS credentials

On startup, the Boto library looks for configuration files in the following locations (in this order):

  • /etc/boto.cfg – for site-wide settings for all users on this machine
  • ~/.boto – for user-specific settings

Use cases

Lets start with a basic scenario:  launching an EC2 instance

#/usr/bin/python
import boto.ec2
conn = boto.ec2.connect_to_region("us-west-2")
conn.run_instances(
    'ami-6ac2a85a',
    key_name='nitheesh_oregon',
    instance_type='t1.micro',
    security_groups=['nitheesh_oregon']
)

The above python script will connect to Amazon’s us-west region and launch an instance using AMI id: ami-6ac2a85a.

Scheduling stop instances. Let’s say you now want to stop your test environment instances at a certain time each day. Run:

#/usr/bin/python
import boto.ec2
conn = boto.ec2.connect_to_region("us-west-2")
conn.stop_instances(instance_ids=['instance-id-1','instance-id-2'])

You can schedule this script to run every day at a particular time by creating a cron job.

CloudFront invalidation. AWS doesn’t provide a command line tool for CloudFront. Therefore, if you want to invalidate objects, you’ll need to log into the console and add the objects to invalidate. It’s a really long and tiresome process. Instead, you can make use of the Boto library:

#/usr/bin/python
import boto
dist_id = 'xxxxxxxxxxx'
invalidation_path = '/home/ec2-user/invalid.txt'
def main():
    paths = open(invalidation_path,"r+")
    conn = boto.connect_cloudfront()
    inval_req = conn.create_invalidation_request(dist_id, paths)
    print inval_req
    touch = open(invalidation_path,"w")
    touch.write("")
if __name__ == '__main__':
    main()

This script reads objects specified in a file, invalidates them, prints an invalidation ID, and clears the contents of invalid.txt.

AWS Services supported by Boto

At the moment, Boto supports more than fifty Amazon services, running the whole range from compute, database, application, and payments and billing. You can find a complete and current list on the Python.org website.

Conclusion 

Boto lets you write scripts that manage complex setups in the Amazon cloud. In addition, it provides support for other public services such as Google Storage, besides private cloud systems like Eucalyptus, OpenStack, and Open Nebula.

If you’re a Python expert and value your time, go for it. If you’re new to Python, Cloud Academy offers an Introduction to Python Learning Path that guides you through the background and basics of Python so you get you the skills and knowledge you need to get started quickly as possible.

For a different way of using Python on AWS, see our blog on Amazon Machine Learning.

Avatar

Written by

Nitheesh Poojary

My professional IT career began nine years back when I was just out of my college. I worked with a great team as an infrastructure management engineer, managing hundreds of enterprise application servers. I found my passion when I got the opportunity to work with Cloud technologies: I'm addicted to AWS Cloud Services, DevOps engineering, and all the cloud tools and technologies that make engineers' lives easier. Currently, I am working as a Solution Architect in SixNines IT. We are an experienced team of engineers that have helped hundreds of customers move to the cloud responsibly. I have achieved 5 AWS certifications, happily helping fellow engineers across the globe through my blogs and answering questions in various forums.

Related Posts

Avatar
Stuart Scott
— June 20, 2019

Working with AWS Networking & Amazon VPC

Being able to architect your own isolated segment of AWS is a simple process using VPCs; understanding how to architect its related networking components and connectivity architecture is key to making it a powerful service.Many services within Amazon Web Services (AWS) require you t...

Read more
  • AWS
  • VPC
Avatar
Stuart Scott
— June 19, 2019

AWS Compute Fundamentals Update

AWS is renowned for the rate at which it reinvents, revolutionizes, and meets customer demands and expectations through its continuous cycle of feature and service updates. With hundreds of updates a month, it can be difficult to stay on top of all the changes made available.  Here ...

Read more
  • AWS
Jeff Hyatt
Jeff Hyatt
— June 18, 2019

10 Steps for an Effective Reserved Instances Strategy

Amazon Web Services (AWS) offers three different ways to pay for EC2 Instances: On-Demand, Reserved Instances, and Spot Instances. This article will focus on effective strategies for purchasing Reserved Instances. While most of the major cloud platforms offer pre-pay and reservation dis...

Read more
  • AWS
  • EC2
Joe Nemer
Joe Nemer
— June 18, 2019

AWS Certification Practice Exam: What to Expect from Test Questions

If you’re building applications on the AWS cloud or looking to get started in cloud computing, certification is a way to build deep knowledge in key services unique to the AWS platform. AWS currently offers 11 certifications that cover major cloud roles including Solutions Architect, De...

Read more
  • AWS
  • AWS Certifications
Avatar
John Chell
— June 13, 2019

AWS Certified Solutions Architect Associate: A Study Guide

The AWS Solutions Architect - Associate Certification (or Sol Arch Associate for short) offers some clear benefits: Increases marketability to employers Provides solid credentials in a growing industry (with projected growth of as much as 70 percent in five years) Market anal...

Read more
  • AWS
  • AWS Certifications
Chris Gambino and Joe Niemiec
Chris Gambino and Joe Niemiec
— June 11, 2019

Moving Data to S3 with Apache NiFi

Moving data to the cloud is one of the cornerstones of any cloud migration. Apache NiFi is an open source tool that enables you to easily move and process data using a graphical user interface (GUI).  In this blog post, we will examine a simple way to move data to the cloud using NiFi c...

Read more
  • AWS
  • S3
Avatar
Chandan Patra
— June 11, 2019

Amazon DynamoDB: 10 Things You Should Know

Amazon DynamoDB is a managed NoSQL service with strong consistency and predictable performance that shields users from the complexities of manual setup.Whether or not you've actually used a NoSQL data store yourself, it's probably a good idea to make sure you fully understand the key ...

Read more
  • AWS
  • DynamoDB
Avatar
Andrew Larkin
— June 6, 2019

The 11 AWS Certifications: Which is Right for You and Your Team?

As companies increasingly shift workloads to the public cloud, cloud computing has moved from a nice-to-have to a core competency in the enterprise. This shift requires a new set of skills to design, deploy, and manage applications in cloud computing.As the market leader and most ma...

Read more
  • AWS
  • AWS Certifications
Sam Ghardashem
Sam Ghardashem
— May 15, 2019

Aviatrix Integration of a NextGen Firewall in AWS Transit Gateway

Learn how Aviatrix’s intelligent orchestration and control eliminates unwanted tradeoffs encountered when deploying Palo Alto Networks VM-Series Firewalls with AWS Transit Gateway.Deploying any next generation firewall in a public cloud environment is challenging, not because of the f...

Read more
  • AWS
Joe Nemer
Joe Nemer
— May 3, 2019

AWS Config Best Practices for Compliance

Use AWS Config the Right Way for Successful ComplianceIt’s well-known that AWS Config is a powerful service for monitoring all changes across your resources. As AWS Config has constantly evolved and improved over the years, it has transformed into a true powerhouse for monitoring your...

Read more
  • AWS
  • Compliance
Avatar
Francesca Vigliani
— April 30, 2019

Cloud Academy is Coming to the AWS Summits in Atlanta, London, and Chicago

Cloud Academy is a proud sponsor of the 2019 AWS Summits in Atlanta, London, and Chicago. We hope you plan to attend these free events that bring the cloud computing community together to connect, collaborate, and learn about AWS. These events are all about learning. You can learn how t...

Read more
  • AWS
  • AWS Summits
Paul Hortop
Paul Hortop
— April 2, 2019

How to Monitor Your AWS Infrastructure

The AWS cloud platform has made it easier than ever to be flexible, efficient, and cost-effective. However, monitoring your AWS infrastructure is the key to getting all of these benefits. Realizing these benefits requires that you follow AWS best practices which constantly change as AWS...

Read more
  • AWS
  • Monitoring