Understanding Nested CloudFormation Stacks

Nested CloudFormation Stack: a guide for developers and system administrators.

AWS’s CloudFormation is widely used by developers and system administrators and it’s not hard to see why. CloudFormation helps you to provision your AWS resources in a predictable order, allowing you to easily configure dependencies and run-time parameters. The collection of AWS resources provisioned by CloudFormation is known as stack. It is written in JSON which makes it very easy to read, understand and share CloudFormation templates. Instead of manually provisioning resources, all you need to do is write a template and launch it with a single click. That’s it. Your AWS stack is up and running with your configuration and your choice of AWS resources. This makes commissioning and decommissioning resources and environments very easy. In case you need a replica environment, you don’t need to re-invent the wheel. You can create multiple copies of your environment using a single CloudFormation template.

However, the focus of this post is not on how to construct a CloudFormation template for provisioning your AWS resources: if you are interested in learning that, we already have a course available for you. This blog post is for developers and system administrators who write CloudFormation templates for automatic infrastructure provisioning. We’ll talk about how they can make their CloudFormation templates more robust, flexible, re-usable, and modular.

What happens when your CloudFormation templates become too complicated?

Let’s take an example scenario:

Jeff is a system administrator/developer working in an organization which has a SaaS product hosted on AWS. This product leverages multiple AWS services like EC2, VPC, S3, RDS, Route53, Autoscaling, and ELB. Also, his organization hosts all their product lifecycle environments, i.e., development, testing, QA, and production on AWS. In order to completely commission and decommission of their various environments, they use AWS CloudFormation service. This simplifies and speeds up the environment creation process. With every new release, Jeff’s organization makes certain environment changes. These changes are mostly related to the application layer. Jeff maintains four different CloudFormation templates, one for each environment. For every environment change, he has to edit all four CloudFormation templates.

Over time, Jeff witnessed CloudFormation growing in size day-by-day and worried as it approached its maximum limit (51,200 bytes) set by AWS. Making changes to templates is also becoming more difficult.

One day, Jeff sat down and had a close look at all four of his templates. He noticed that 70% of the work performed by these templates is overlapping, and, as they evolve, the majority of the architecture isn’t even touched. He started looking for ways to minimize the size of his CloudFormation templates and to improve their usability. After researching for a few hours, he found “Nested Stacks.”

Nested CloudFormation stacks

Nested stacks are AWS CloudFormation stacks inside stacks. These nested stacks under the main stack are considered as a resource. To more easily understand, let’s call child templates “functions,” and let our main stack act as an entry point for calling those functions.

Jeff started by identifying those portions of his templates that were repeated and began building separate templates for those portions. For a master template which nests other child templates, he made use of AWS::CloudFormation::Stack resource type.

The syntax of AWS::CloudFormation::Stack is

{
   "Type" : "AWS::CloudFormation::Stack",
   "Properties" : {
     "NotificationARNs" : [ String, ... ],
     "Parameters" : { CloudFormation Stack Parameters Property Type },
     "TemplateURL" : String,
     "TimeoutInMinutes" : String
   }
}

Where:

  • NotificationARNs is a list of existing Amazon SNS topics where notifications about stack events are sent.
  • TemplateURL is the URL of the template that specifies the stack you want to create as a resource. This template must be stored in an Amazon S3 bucket.
  • Parameters are a set of parameters passed when this nested stack is created.
  • TimeoutInMinutes is the time AWS CloudFormation waits for the nested stack to reach the CREATE_COMPLETE state.

Nested stacks also allowed him to pass any parameters declared in his master template to child templates by using the Parameters properties as shown above. It also allowed him to reference outputs of one child stack as a parameter in another child stack by using Fn::GetAtt intrinsic function. For example:

{
   "AWSTemplateFormatVersion": "2010-09-09",
   "Resources": {
       "ChildStack01": {
           "Type": "AWS::CloudFormation::Stack",
           "Properties": {
               "TemplateURL": "https://s3.amazonaws.com/cloudformation-templates-us-east-1/VPC.template",
               "TimeoutInMinutes": "60"
           }
       },
       "ChildStack02": {
           "Type": "AWS::CloudFormation::Stack",
           "Properties": {
               "TemplateURL": "https://s3.amazonaws.com/cloudformation-templates-us-east-1/Subnet.template",
               "Parameters": {
                  "VpcId" : { "Fn::GetAtt" : [ "ChildStack01", "Outputs.VpcID" ] },
               },
               "TimeoutInMinutes": "60"
           }
       }
   },
   "Outputs": {
       "StackRef": {
           "Value": { "Ref": "ChildStack02" }
       },
       "OutputFromNestedStack": {
           "Value": { "Fn::GetAtt": [ "ChildStack02", "Outputs.SubnetID" ]}
       }
   }
}

In the above example, we see ChildStack01’s output VpcID was passed as a parameter to ChildStack02, and towards the end, ChildStack02’s output SubnetID was passed to Outputs.

CloudFormation and dependencies

However, Jeff was still worried about parallel provisioning of resources as there are some environment dependencies, but later he found that wherever there was a dependency that one child stack should come up before another child stack, he can use DependsOn attribute or GetAtt function. CloudFormation is smart enough to identify dependencies while paralleling other resources.
Finally, he wanted an easy approach to update his stack and found that AWS also provides an ability to update a top-level stack and its nested stacks together. Whenever any changes are made to any template, CloudFormation detects the updates made to the top-level template as well as the nested templates and will update only those stacks whose templates have changed.

Working with a nested template made Jeff’s life much easier. His CloudFormation templates are far more organized, modular and re-usable. Whenever there is any change in any layer of his architecture, he selects that particular template and makes the necessary changes. For the 70% of his configurations that are common across all product lifecycle environments, he’ll just have to call those portions as child templates. This allows him to focus on updates made to the environment rather than maintaining and replicating changes in all environments.

Avatar

Written by

Sanket Dangi

Head of Managed Services at REAN Cloud. Before joining REAN Cloud, I was CEO and Founder of StraightArc Solutions which was later acquired by REAN Cloud. I started my career working on cloud computing. Loves to talk about DevOps, System Administration, Scalability, High Availability, Disaster Recovery and Cloud Security. Apart from work, I love to meet people, travel and watch sports.


Related Posts

Avatar
Sanket Dangi
— February 11, 2019

WaitCondition Controls the Pace of AWS CloudFormation Templates

AWS's WaitCondition can be used with CloudFormation templates to ensure required resources are running. As you may already be aware, AWS CloudFormation is used for infrastructure automation by allowing you to write JSON templates to automatically install, configure, and bootstrap your ...

Read more
  • AWS
  • CloudFormation
Avatar
Paul Carlstroem
— March 2, 2016

Advanced Use of AWS CloudFormation

Advanced Use of AWS CloudFormation is a fantastic new course from a superb instructor Our exceptional AWS course content continues growing with the addition of  Advanced Use of AWS CloudFormation. This is an 8 lesson course with 2 hours of video content designed, built, and presented b...

Read more
  • AWS
  • CloudFormation
Avatar
Rafael Nunes
— January 11, 2016

Publish a SharePoint Farm Using CloudFormation

We'll show you how to automate and deploy a SharePoint Farm fast using CloudFormation the right way. Have you ever needed to quickly create a brand new SharePoint Farm and give the users access through the internet? Like when your client says: "Hey Rafael, we just watched all the Share...

Read more
  • AWS
  • Azure
  • CloudFormation
Avatar
Chandan Patra
— August 12, 2015

DevOps in AWS: CloudFormation, CodeCommit, CodePipeline, and CodeDeploy

We are entering a new era of technology this is causing and a cultural shift in the way software projects are built. The waterfall model paved the way for agile development a few years back. Today a need exists for better collaboration between IT operations, developers, and IT Infrastru...

Read more
  • AWS
  • CloudFormation
  • DevOps
Avatar
Christian Petters
— July 15, 2015

CloudFormation and AWS Deployment Automation

CloudFormation, OpsWorks, and Elastic Beanstalk: why you can (almost) never apply too much automation to your deployment management. If you are like me (and I know that many of you are), then I am sure this has happened to you. You get your hands on a new toy (like CloudFormation) and ...

Read more
  • AWS
  • CloudFormation
Avatar
Michael Sheehy
— June 30, 2015

Is CloudFormation the Perfect Cloud Deployment Tool for AWS?

AWS CloudFormation lets you create and provision AWS infrastructure deployments predictably and repeatedly using templates. Cloud deployment, and specifically AWS cloud deployment, can be a daunting task and AWS provides many us with many useful tools. However, this time I am going to ...

Read more
  • AWS
  • CloudFormation
Avatar
Michael Sheehy
— May 21, 2015

Writing Your First CloudFormation Template

What you'll need to write your first CloudFormation template AWS CloudFormation is a service that helps you model and set up your Amazon Web Services resources so you can spend less time managing those resources, and more time focusing on your applications. CloudFormation can: Sim...

Read more
  • CloudFormation
Avatar
Andrea Colangelo
— September 8, 2014

Learn AWS Automation And How to Use CloudFormation With Our New Course

I'm so glad to see a brand new course available in the Cloud Academy Training Library, and an extremely interesting one too How to Use CloudFormation for AWS Automation, made by our Senior DevOps Engineer Antonio Angelino, who worked hard to craft this new tutorial about one of the most...

Read more
  • AWS
  • CloudFormation