The course is part of this learning path
Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can help with multi-cloud by having one workflow for all clouds. The infrastructure Terraform manages can be hosted on public clouds like Amazon Web Services, Microsoft Azure, and Google Cloud Platform, or on-prem in private clouds such as VMWare vSphere, OpenStack, or CloudStack. Terraform treats infrastructure as code (IaC) so you never have to worry about you infrastructure drifting away from its desired configuration. If you like what you are hearing about Terraform then this course is for you!
In this course, we’ll learn Terraform from the ground up. While building a strong foundation for you to solve real-world challenges with Terraform, you'll learn about its core concepts including HashiCorp Configuration Language, providers, resources, and state. The course concludes with a demo to illustrate how Terraform can be used to manage a practical infrastructure for deploying development and production versions of a two-tier application in Google's Cloud using Cloud SQL, Google Kubernetes Engine (GKE), and Kubernetes. The Terraform configuration files used in the course are all available in the course's GitHub repository.
This course is for anyone that is interested in managing infrastructure in public, private, or hybrid clouds. Some roles that fit into that category are:
- DevOps Engineers
- IT Professionals
- Cloud Engineers
After completing this course, you will be able to:
- Describe what Terraform is
- Write Terraform configuration files
- Understand how Terraform integrates infrastructure sources
- Manage multiple infrastructure environments with Terraform
This is an intermediate level course that assumes:
- You have priors experience with a scripting or programming language.
|Lesson||What you'll learn|
|Introduction||What will be covered in this course|
|What is Terraform?||Take a high-level look at what Terraform is and when to use it|
|Terraform Configuration||Understand the ins and outs of HashiCorp Configuration Language (HCL)|
|Providers||Discover how Terraform integrates various infrastructure sources|
|Resources||See how to configure parameters that are common to all resources|
|State||Learn how Terraform state connects your configuration with the real world|
|Two-Tier App Demo||See how to deploy a two-tier app in multiple environments with Terraform|
|Summary||Review the course and see what's next|
If you have thoughts or suggestions for this course, please contact Cloud Academy at firstname.lastname@example.org.
About the Author
Logan has been involved in software development and research since 2007 and has been in the cloud since 2012. He is an AWS Certified DevOps Engineer - Professional, AWS Certified Solutions Architect - Professional, Microsoft Certified Azure Solutions Architect Expert, MCSE: Cloud Platform and Infrastructure, Google Cloud Certified Associate Cloud Engineer, Certified Kubernetes Administrator (CKA), Certified Kubernetes Application Developer (CKAD), Linux Foundation Certified System Administrator (LFCS), and Certified OpenStack Administrator (COA). He earned his Ph.D. studying design automation and enjoys all things tech.
This lesson will give a high-level overview of Terraform for our first look at the subject. After this lesson, you’ll be able to answer questions like what is Terraform, why would you use it, and a simplified version of how does it work?
We’ll start by defining Terraform and the space that it occupies.
Next, we’ll look into what specifically Terraform can do for you.
Lastly, we’ll see different ways of how you can use Terraform.
Let’s start with a definition. Terraform is defined by HashiCorp, the creator of Terraform, as a tool for building, changing, and versioning infrastructure safely and efficiently. That sounds like an infrastructure as code solution, because Terraform is an infrastructure as code solution.
Infrastructure as code
Infrastructure as code, or IaC, has gained a lot of momentum over the years because helps solves several problems that plagued infrastructure management in the past:
· Reproducible environments: By using code to generate infrastructure, the same environment can be created over and over. Over time an environment can drift away from its desired state and difficult to diagnose issues can creep into your release pipeline. With IaC no environment gets special treatment and fresh new environments are easily created and destroyed.
· Idempotence & Convergence: Extending the last point, idempotence is the trait that no matter you apply the configuration described by your IaC, there are no side effects on the environment. Convergence is the trait that actions are only taken if they need to be. In IaC, only the actions needed to bring the environment to the desired state are executed. If the environment is already in the desired state, no actions are taken.
· Easing collaboration: Having the code in a version control system like Git allows teams to collaborate on infrastructure. Team members can get specific versions of the code and create their own environments for testing or other scenarios.
· Self-service infrastructure: A pain point that often existed for developers before moving to cloud infrastructure was the delays required to have operations teams create the infrastructure they needed to build new features and tools. With the elasticity of the cloud allowing resources to be created on-demand, developers can provision the infrastructure they need when they need it. IaC further improves the situation by allowing developers to use infrastructure modules to create identical environments at any point in the application development lifecycle. The infrastructure modules could be created by operations and shared with developers freeing developers from having to learn another skill.
All combined these benefits make IaC a staple in DevOps practices.
Some Other IaC Options
Cloud providers usually provide their own native infrastructure as code solution. For example:
· Amazon Web Services has CloudFormation,
· Microsoft Azure has Azure Resource Manager templates, and
· Google Cloud Platform has deployment manager.
Each comes with their own unique way of describing infrastructure in code. That also holds for private clouds like OpenStack,
which has Heat for describing infrastructure in code. With the trend in industry tending to move toward embracing multi-cloud, including hybrid clouds, it is more efficient to have one tool and one common workflow to manage infrastructure no matter where it is hosted. Even if you are only using one cloud now, it may be worth futureproofing yourself in case you do leverage multiple clouds later on.
Terraform can help with multi-cloud by having one workflow for all clouds. The infrastructure Terraform manages can be hosted on public clouds like Amazon Web Services, Azure, and Google Cloud Platform, or on-prem in private clouds such as OpenStack, VMWare vSphere, or CloudStack. Terraform infrastructure integrations also allow you manage software and services, including databases like MySQL, source control systems like GitHub, configuration management tools like Chef, and much more. There are over 100 publicly available infrastructure integrations. It is very powerful being able to describe all of your infrastructure in Terraform. Even if you are only using one cloud, Terraform is a great option and can make it easier leverage multi-cloud later.
When to Use Terraform?
By now you have a good idea of what you can accomplish with Terraform. To reinforce when you should consider using Terraform, let’s consider some examples use cases highlighted by HashiCorp.
The first we’ll consider is disposable environments. It is common practice to have both a production and staging or QA environment. These environments are smaller clones of their production counterpart, but are used to test new applications before releasing in production. As the production environment grows larger and more complex, it becomes increasingly onerous to maintain an up-to-date staging environment.
Using Terraform, the production environment can be codified and then shared with staging, QA or dev. These configurations can be used to rapidly spin up new environments to test in, and then be easily disposed of. Terraform can help tame the difficulty of maintaining parallel environments, and makes it practical to elastically create and destroy them.
Another example use case for Terraform is multi-cloud deployment. It's can be attractive to spread infrastructure across multiple clouds to increase fault-tolerance. By using only a single region or cloud provider, fault tolerance is limited by the availability of that provider. Having a multi-cloud deployment allows for more graceful recovery of the loss of a region or entire provider. Increasing fault-tolerance also applies for hybrid clouds where you have your own private cloud running on-prem but also leverage a public cloud for business continuity and disaster recovery. As examples, backing up data, tiering access to data, or running a secondary database or application in-case your primary on-prem servers fail. You may also encounter situations where one public cloud provider has a service that isn’t’ available on your preferred public provider.
Realizing multi-cloud deployments can be very challenging as many existing tools for infrastructure management are cloud-specific. Terraform is cloud-agnostic and allows a single configuration to be used to manage multiple providers, and to even handle cross-cloud dependencies. This simplifies management and orchestration, helping operators build large-scale multi-cloud infrastructures.
Multi-tier applications are another example of where Terraform can shine. A very common pattern is the N-tier architecture. The most common 2-tier architecture is a pool of web servers that use a database tier. Additional tiers get added for API servers, caching servers, routing meshes, etc. This pattern is used because the tiers can be scaled independently and provide a separation of concerns.
Terraform is an ideal tool for building and managing these infrastructures. Each tier can be described as a collection of resources, and the dependencies between each tier are handled automatically; Terraform will ensure the database tier is available before the web servers are started and that the load balancers are aware of the web nodes. Each tier can then be scaled easily using Terraform by modifying a single count configuration value. Because the creation and provisioning of a resource is codified and automated, elastically scaling with load becomes trivial.
The last use case that I’ll mention is resource schedulers. In large-scale infrastructures, static assignment of applications to machines becomes increasingly challenging. To solve that problem, there are a number of schedulers like Borg, Mesos, YARN, and Kubernetes. These can be used to dynamically schedule Docker containers, Hadoop, Spark, and many other software tools.
Terraform is not limited to physical providers like AWS. Resource schedulers can be treated as a provider, enabling Terraform to request resources from them. This allows Terraform to be used in layers: to setup the physical infrastructure running the schedulers as well as provisioning onto the scheduled grid.
You can find these use cases and more on Terraforms website https://www.terraform.io/intro/use-cases.html.
Terraform Product Streams
Terraform can refer to different products and tiers.
· The core product is the open source version of Terraform. It includes all the pieces needed to provide infrastructure as code and includes support of the over 100 infrastructure integrations discussed earlier. The source code of the open source Terraform product lives on GitHub and is written in the Go language. HashiCorp hosts compiled open source releases as a single zipped binary on the terraform.io website. The open source version provides a command-line interface for managing your infrastructure.
· The other product is Terraform Enterprise, which is available in two tiers. Both tiers include the features of the open source product and add additional features useful for larger teams.
o Pro is the first tier. Pro is software as a service that runs in the cloud and is managed by HashiCorp. The pro tier includes a graphical user interface, version control connections to modify infrastructure as changes are committed, API access to integrate terraform into existing tooling, and a private infrastructure module registry for sharing re-useable modules within your organization. You also receive business hour support from HashiCorp with the Pro tier.
o Premium is the second enterprise tier. Premium is installed on your own cloud infrastructure, that currently must be in AWS. It includes the features of Pro as well as policy as code, single sign on with SAML, and provides audit logs of every infrastructure change. You also receive 24x7 support gold-level support from HashiCorp.
This course focuses on the open source version of Terraform. The enterprise versions builds upon the open source version, so what you learn in this Course also applies to the enterprise version. From here this point onward in the Course, when I mention Terraform I’ll be talking about the open source product, unless otherwise indicated.
The open source Terraform product compiles to a tool that you interact with on the command-line. As an example, this screenshot shows a snippet of the output when you run terraform at the command-line. The command-line tool uses configuration files that specify your infrastructure as code.
Getting the CLI
I mentioned that HashiCorp hosts downloads of the releases of the open source Terraform binaries. You can find them at terraform.io/downloads.html. The link is provided at the end of this lesson’s transcript. The downloads page will always have the latest version of Terraform. HashiCorp also hosts previous release versions for all of the platforms. You can use the version that works for you and upgrade when you choose to.
For the sake of following along with this course, a link to the version used in this Course is provided at the bottom of this lesson’s transcript.
Because Terraform is a single binary, there isn’t an installer. You only need to unzip the compressed archives HashiCorp hosts on the Terraform website and place the binary in a directory in your environment’s PATH or add Terraform’s directory to your PATH.
You know that there is a CLI and configuration files, but we haven’t discussed a typical workflow for managing infrastructure with Terraform. This slide will illustrate how you would use Terraform when you are developing infrastructure code.
First, you write or modify configuration files for your infrastructure. The configuration files declare the desired state of your infrastructure. You modify existing configuration files to declare how you want to change existing infrastructure. All of the remaining lessons in this Course illustrate writing and working with configuration files, but the following lesson is entirely devoted to the subject.
Once you are happy with your declared configuration, you can ask Terraform to generate an execution plan for it. The plan command in the CLI is used to generate an execution plan from a configuration. The execution plan tells you what changes Terraform would need to make to bring your current infrastructure to the declared state in your configuration. Because Terraform is convergent, it will plan the fewest required actions to bring the infrastructure to the desired configuration. Terraform also considers dependencies to determine the order that changes must be applied in. The plan stage is relatively inexpensive compared to actually applying changes, so you can liberally use the plan command while developing your configuration to see what changes would need to take place. For complex infrastructures the plan stage can take a long time. This is because to correctly plan changes, terraform must get the current state of all the components in your infrastructure by making API requests. There are ways to work around slow plan generation if you ever encounter this situation. You can tell Terraform to only target a subset of the infrastructure or by telling Terraform to use the stale state information for a plan.
Based on the required changes you see in the plan, you can either accept or reject the changes. By separately planning changes before apply them, Terraform allows you to avoid mistakes and uncertainty.
If you don’t like what you see in the plan, then you can go back and modify your configuration without having done any harm.
If you accept the plan you can instruct Terraform to apply changes. You use the apply command to accomplish this. The apply command can use the plan that you generate with the plan command. If you don’t provide a plan, apply can generate one and ask for you to accept the plan before applying the changes. Terraform will make the API calls required to implement the changes. If anything goes wrong terraform will not attempt to automatically rollback the infrastructure to the state it was in before running apply. This is because apply adheres to the plan. It won’t delete a resource if the plan doesn’t call for it. If you version control your infrastructure configuration code, and I strongly encourage you to, you can use a previous version of your configuration to roll back. Alternatively, you can use the destroy or taint command to target components that need to be deleted or recreated, respectively.
I mentioned that Terraform uses dependency information to plan the order of infrastructure changes. Terraform builds a resource graph that captures all the dependency information in your infrastructure. The dependencies are usually naturally expressed through the configuration and don’t require you to go through marking what depends on what. Using the resource graph, Terraform can apply changes in parallel when no dependencies exist and builds your infrastructure as efficiently as possible. Terraform can output the graph and you can visualize it to understand more about your infrastructure. The example shows an example from the Manage AWS Resources with Terraform Lab here on Cloud Academy. The graph shows the dependencies between an instance, subnets, and, a VPC. You’ll learn more about the resource graph when you do the Lab.
The typical workflow I’ve described is when you are manually working on your infrastructure’s configuration.
However, Terraform supports automation, for example by integrating into continuous integration or continuous deployment pipelines. You may automatically create infrastructure to deploy new branches
If you decide it’s appropriate, you can automatically accept plans to have infrastructure update to the latest version without any human intervention. That’s probably not a good idea for production environments.
You can instead opt to keep the manual approval step. With the generated execution plan and resource graph, it is easy to understand what is going to change and in what order. Even for complex changesets, the chance of applying making unintentional changes is greatly reduced.
Lastly, you can also generate a plan only. This can be useful for reviewing pull requests. You can quickly see the implications of the changes on your current infrastructure by inspecting the plan.
Migrating to Terraform
If you like what you are hearing about Terraform but have already invested in another infrastructure as code solution,
Terraform can help transfer control of your infrastructure to Terraform. Terraform supports importing existing resources to bring them under its control using the import command. Importing resources won’t be covered in more detail in this Course, but you can try out importing for yourself in the Manage AWS Resources with Terraform Lab. In that Lab you’ll start by importing a resource managed by AWS CloudFormation. If you are taking this course as part of a Terraform Learning Path, then the Labs are coming next after you complete this Course.
This lesson covered Terraform from several angles to help you answer the what, why, and when questions related to Terraform.
We started by reviewing the benefits of infrastructure as code and mentioned several options for infrastructure as code solutions for different clouds.
Terraform can be used to orchestrate infrastructure changes across several public and private cloud providers. It can also be used to manage software and services making it a diverse infrastructure tool.
We discussed the different Terraform products, but we’ll focus on the open source product as the common thread across all products.
Lastly, we covered some workflows that Terraform supports running the spectrum from manual to fully automated.
This lesson covered Terraform at high-level. In the following lessons we’ll start to dig deeper into different aspects of Terraform, starting with the ins and outs of configurations in Terraform. Continue on to the next lesson when you’re ready to learn how to write Terraform configurations.
Latest open source Terraform release download page: https://terraform.io/downloads.html
Download page for the release used in this Course: https://releases.hashicorp.com/terraform/0.11.3/