Cloud Academy Team

Understanding Nested CloudFormation Stacks

Nested CloudFormation Stack: a guide for developers and system administrators.

AWS’s CloudFormation is widely used by developers and system administrators and it’s not hard to see why. CloudFormation helps you to provision your AWS resources in a predictable order, allowing you to easily configure dependencies and run-time parameters. The collection of AWS resources provisioned by CloudFormation is known as stack. It is written in JSON which makes it very easy to read, understand and share CloudFormation templates. Instead of manually provisioning resources, all you need to do is write a template and launch it with a single click. That’s it. Your AWS stack is up and running with your configuration and your choice of AWS resources. This makes commissioning and decommissioning resources and environments very easy. In case you need a replica environment, you don’t need to re-invent the wheel. You can create multiple copies of your environment using a single CloudFormation template.

However, the focus of this post is not on how to construct a CloudFormation template for provisioning your AWS resources: if you are interested in learning that, we already have a course available for you. This blog post is for developers and system administrators who write CloudFormation templates for automatic infrastructure provisioning. We’ll talk about how they can make their CloudFormation templates more robust, flexible, re-usable, and modular.

What happens when your CloudFormation templates become too complicated?

Let’s take an example scenario:

Jeff is a system administrator/developer working in an organization which has a SaaS product hosted on AWS. This product leverages multiple AWS services like EC2, VPC, S3, RDS, Route53, Autoscaling, and ELB. Also, his organization hosts all their product lifecycle environments, i.e., development, testing, QA, and production on AWS. In order to completely commission and decommission of their various environments, they use AWS CloudFormation service. This simplifies and speeds up the environment creation process. With every new release, Jeff’s organization makes certain environment changes. These changes are mostly related to the application layer. Jeff maintains four different CloudFormation templates, one for each environment. For every environment change, he has to edit all four CloudFormation templates.

Over time, Jeff witnessed CloudFormation growing in size day-by-day and worried as it approached its maximum limit (51,200 bytes) set by AWS. Making changes to templates is also becoming more difficult.

One day, Jeff sat down and had a close look at all four of his templates. He noticed that 70% of the work performed by these templates is overlapping, and, as they evolve, the majority of the architecture isn’t even touched. He started looking for ways to minimize the size of his CloudFormation templates and to improve their usability. After researching for a few hours, he found “Nested Stacks.”

Nested CloudFormation stacks

Nested stacks are AWS CloudFormation stacks inside stacks. These nested stacks under the main stack are considered as a resource. To more easily understand, let’s call child templates “functions,” and let our main stack act as an entry point for calling those functions.

Jeff started by identifying those portions of his templates that were repeated and began building separate templates for those portions. For a master template which nests other child templates, he made use of AWS::CloudFormation::Stack resource type.

The syntax of AWS::CloudFormation::Stack is

{
   "Type" : "AWS::CloudFormation::Stack",
   "Properties" : {
     "NotificationARNs" : [ String, ... ],
     "Parameters" : { CloudFormation Stack Parameters Property Type },
     "TemplateURL" : String,
     "TimeoutInMinutes" : String
   }
}

Where:

  • NotificationARNs is a list of existing Amazon SNS topics where notifications about stack events are sent.
  • TemplateURL is the URL of the template that specifies the stack you want to create as a resource. This template must be stored in an Amazon S3 bucket.
  • Parameters are a set of parameters passed when this nested stack is created.
  • TimeoutInMinutes is the time AWS CloudFormation waits for the nested stack to reach the CREATE_COMPLETE state.

Nested stacks also allowed him to pass any parameters declared in his master template to child templates by using the Parameters properties as shown above. It also allowed him to reference outputs of one child stack as a parameter in another child stack by using Fn::GetAtt intrinsic function. For example:

{
   "AWSTemplateFormatVersion": "2010-09-09",
   "Resources": {
       "ChildStack01": {
           "Type": "AWS::CloudFormation::Stack",
           "Properties": {
               "TemplateURL": "https://s3.amazonaws.com/cloudformation-templates-us-east-1/VPC.template",
               "TimeoutInMinutes": "60"
           }
       },
       "ChildStack02": {
           "Type": "AWS::CloudFormation::Stack",
           "Properties": {
               "TemplateURL": "https://s3.amazonaws.com/cloudformation-templates-us-east-1/Subnet.template",
               "Parameters": {
                  "VpcId" : { "Fn::GetAtt" : [ "ChildStack01", "Outputs.VpcID" ] },
               },
               "TimeoutInMinutes": "60"
           }
       }
   },
   "Outputs": {
       "StackRef": {
           "Value": { "Ref": "ChildStack02" }
       },
       "OutputFromNestedStack": {
           "Value": { "Fn::GetAtt": [ "ChildStack02", "Outputs.SubnetID" ]}
       }
   }
}

In the above example, we see ChildStack01’s output VpcID was passed as a parameter to ChildStack02, and towards the end, ChildStack02’s output SubnetID was passed to Outputs.

CloudFormation and dependencies

However, Jeff was still worried about parallel provisioning of resources as there are some environment dependencies, but later he found that wherever there was a dependency that one child stack should come up before another child stack, he can use DependsOn attribute or GetAtt function. CloudFormation is smart enough to identify dependencies while paralleling other resources.
Finally, he wanted an easy approach to update his stack and found that AWS also provides an ability to update a top-level stack and its nested stacks together. Whenever any changes are made to any template, CloudFormation detects the updates made to the top-level template as well as the nested templates and will update only those stacks whose templates have changed.

Working with a nested template made Jeff’s life much easier. His CloudFormation templates are far more organized, modular and re-usable. Whenever there is any change in any layer of his architecture, he selects that particular template and makes the necessary changes. For the 70% of his configurations that are common across all product lifecycle environments, he’ll just have to call those portions as child templates. This allows him to focus on updates made to the environment rather than maintaining and replicating changes in all environments.

Cloud Academy