Codify your entire infrastructure with Terraform

Terraform lets you unify your infrastructure by using infrastructure as code (IaC) to create and manage your entire infrastructure stack. While organizations use Terraform to manage traditional infrastructure resources (for example, networking, servers, and databases), you can extend Terraform to manage your entire infrastructure stack using a single configuration language, tool, and implementation pattern. This may include vendors outside of traditional cloud providers such as vendors for CI/CD, observability, and databases. By adopting Terraform for your entire stack, your organization can consistently manage resources, minimize errors, and enhance deployment agility.

The first section of this document explains the benefits of adopting Terraform and how it enables consistent workflows to manage your resources. The remaining sections dive into each provider category in the Terraform Registry so you can fully adopt Terraform to streamline your organization. These sections are not an exhaustive list of all the Terraform providers.

The provider types covered in this document are:

• Public cloud and Cloud automation
• Container orchestration
• Continuous Integration/Deployment (CI/CD)
• Logging & Monitoring
• Database
• Version control systems
• Security and authentication

Terraform-driven development

Terraform uses providers to directly interact with cloud resources and services. Terraform creates and manages these resources by storing the state of your infrastructure. Terraform codifies cloud APIs into declarative configuration files, using HashiCorp Configuration Language (HCL). You define the desired state of your infrastructure in HCL and Terraform will deploy and configure the resources to match your configuration.

In addition to configuring your networks, servers, and databases, you can also use Terraform to manage other resources like containers, machine images, and observability services. Terraform enables you to create an infrastructure delivery workflow where you create consistent infrastructure, application configurations, and images. This workflow consists of adopt, build, standardize, and scale.

• Adopt: Compose infrastructure as code in a Terraform file using HCL to provision resources from any infrastructure provider.
• Build: Infrastructure automation workflows to compose, collaborate, reuse, and provision infrastructure as code across IT operations and teams of developers.
• Standardize: Establish guardrails for security, compliance, and cost management through role-based access controls, policy, enforcement, and audit.
• Scale: Extend workflow automation to all teams in the organization with self-service infrastructure as code and integrate with VSC, ITSM, and CI/CD.

The Terraform Registry is a single source for finding providers and their supporting documentation. You can find official, partner, and community providers, modules, policy libraries, and run tasks in the Terraform Registry. The Terraform Registry is a great first step in learning all the resources you can manage with Terraform.

Modules are the main way to package and reuse resource configurations with Terraform. This is especially useful for standardizing your infrastructure deployments. You can find publicly available Terraform modules for configuring many kinds of common infrastructure in the Terraform Registry. These modules are free to use, and Terraform can download them automatically if you specify the appropriate source and version in a module call block.

Your organization may produce modules tailored for your own infrastructure needs. HCP Terraform and Terraform Enterprise both include a private module registry for sharing modules internally within your organization.

HashiCorp resources:

• Terraform providers registry
• Terraform providers documentation
• Learn to write your own Terraform provider
• Learn about Terraform modules
• Learn HCL by writing Terraform configuration
• Learn HCL by following our Terraform certification prep

Use policy as code

When you provision infrastructure, manage secrets, and manage your services, you want to ensure they follow certain behaviors. By managing policy with infrastructure as code, you can unify the process of maintaining and implementing policies across your entire infrastructure stack. This approach brings consistency, automation, and version control to policy management.

HCP Terraform uses Sentinel to enable granular policy control for your infrastructure. Sentinel is a language and policy framework, which restricts Terraform actions to defined, allowed behaviors. Policy authors manage Sentinel policies in HCP Terraform with policy sets, which are groups of policies. Organization owners control the scope of policy sets by applying certain policy sets to the entire organization or by selecting workspaces.

A common Sentinel policy is to help control cloud cost. By using these policies, in combination with HCP Terraform, you can ensure your IaC doesn’t create infrastructure that isn't appropriate for your budget.

The following is an example of a policy that limits the creation of instances outside of the three listed instance types.

import "tfplan"

main = rule {
  all tfplan.resources.aws_instance as _, instances {
    all instances as _, r {
    r.applied.instance_type in ["t2.micro", "t2.small", "t2.medium"]
    }
  }
}

When organizations use Sentinel in HCP Terraform, they can ensure that all infrastructure changes adhere to corporate policies, security standards, and compliance requirements. This proactive approach to policy enforcement helps prevent misconfigurations, reduces security risks, and maintains consistency across the infrastructure lifecycle.

You can also enforce OPA policies with HCP Terraform. These policies, written in Rego, can validate resource configurations, enforce tagging standards, control costs, and ensure adherence to company or regulatory requirements. Terraform runs OPA policy checks before each plan or apply, preventing implementation of non-compliant changes.

HashiCorp resources:

• HCP Sentinel
• Get started with Sentinel
• Sentinel language
• Define OPA policies with HCP Terraform

Public cloud and Cloud automation

A good starting point when unifying your cloud infrastructure is understanding all the resources Terraform can manage in your cloud. The best way to do this is to review the cloud provider resources located in the Terraform Registry.

Terraform integrates into most major clouds, including AWS, Azure, GCP, Oracle, VMWare, and OVHcloud. These cloud providers give you the resources to create a full cloud stack that’s ready to run your application. Cloud automation providers give you specialized cloud functions, and generally will not provide you with all the resources that the major cloud providers will. These providers include Helm, F5, Ansible, and more. A common pattern is to review the cloud documentation to determine what cloud infrastructure you want to deploy, and then refer to the provider documentation to learn which resource you should use.

You can also visit the cloud provider modules. These modules give you pre-written code that you can use to deploy your infrastructure. For example, you can use the AWS S3-bucket module to deploy an S3 bucket by only passing in the required variable and authenticating to AWS.

HashiCorp resources:

• List of public cloud and cloud automation providers in the Terraform Registry.
• Get started tutorials for AWS, Azure, and Google Cloud
• Provision infrastructure with Terraform Cloud-Init

Container orchestration

Terraform can automate the provisioning, configuration, and management of orchestration systems. These systems range from self-managed and cloud-managed orchestrators like Nomad and Kubernetes to cloud-native container services like AWS ECS.

We recommend you separate the configuration to deploy, configure, and manage the orchestrator from the configuration to deploy services to the orchestrator. For example, you would have Terraform deploy your Kubernetes system in one configuration and then deploy services like NGINX in a different configuration.

Kubernetes

Terraform allows you to manage Kubernetes as infrastructure as code. It can create and manage cloud resources such as virtual machines, networks, load balancers, and storage systems across various cloud providers like AWS, GCP, and Azure. For example, on AWS, Terraform can set up an EKS (Elastic Kubernetes Service) cluster, configure VPCs, subnets, and security groups, and provision EC2 instances to serve as worker nodes.

Terraform can interact directly with Kubernetes to manage cluster resources. Terraform can use the Kubernetes provider to create and manage namespaces, deploy workloads, set up services, and configure ingress rules. It can also handle more complex Kubernetes concepts like role-based access control (RBAC) by enabling you to set up roles, role bindings, and service accounts.

While self-managed Kubernetes is the Container orchestration category in the Terraform Registry, the cloud-managed versions are in the cloud-specific provider documentation. For example, the AWS EKS (Elastic Kubernetes Service) documentation is in the AWS provider documentation under aws_eks_cluster.

Benefits of using Terraform:

• Deploy orchestrator to multiple environments
• Manage container workloads
• Manage orchestrator configuration with IaC
• Manage hardware orchestrator runs on

HashiCorp resources:

• List of container orchestration providers in the Terraform Registry
• Tutorials to manage Kubernetes with Terraform
• Deploy cloud native Kubernetes , EKS, AKS, and GKE
• Deploy federated multi-cloud Kubernetes clusters
• Manage Kubernetes resources via Terraform
• Deploy applications with the Helm provider
• HCP Terraform Operator for Kubernetes overview

Continuous integration/deployment (CI/CD)

When you use Terraform to manage your CI/CD, you are able to manage the CI/CD system’s configuration through version control and create a defined approval process for system modifications.

An example is managing CI/CD build agents. CI/CD systems use build agents to run jobs like building software, or deploying infrastructure. CI/CD tools, like GitHub and GitLab, often offer build agents as a service, meaning they manage the build agent for you. However, you can manage the build agents yourself, which gives you control of hardware, operating system, and software tools. Packer can create an image and install the agent and other software. The build agent image can be a VM image or a container. Terraform can then create the build infrastructure, whether it’s VMs or container clusters, and deploy your build agents.

Terraform can manage other components of your CI/CD systems. As CI/CD systems are vital to your organization's operational excellence, managing these systems as code will increase their reliability and security by adding a defined approval process audit logs for configuration changes.

Benefits of using Terraform:

• Configure build agents and build infrastructure
• Setup CI/CD in dev, staging, and production environments
• Manage security and IAM as code
• Manage build tokens and keys
• Create approval process and audit logs for configuration changes

HashiCorp resources:

• List of CI/CD providers in the Terraform Registry
• Terraform GitLab and GitHub providers, with documentation to manage build agents
• Terraform GitHub Actions

External resources:

• Learn how to manage BuildKite CI/CD with Terraform

Logging and monitoring

You can use Terraform to deploy and manage logging and monitoring services. Instead of creating scripts to interact with monitoring tools APIs, you can directly configure monitoring solutions like Datadog and Grafana with their Terraform provider. Terraform uses monitoring as code (MaC) to install and set up monitoring agents to collect metrics and logs of your infrastructure and applications, and create dashboards to view the metrics and set up alerts to notify you of issues. MaC gives you all of the benefits of infrastructure as code, such as version control and automated deployments, increasing your organization’s reliability and meeting RTO and RPO expectations.

Benefits of using Terraform:

• Use monitoring as code
• Create dashboards and alerts with infrastructure as code

HashiCorp resources:

• List of logging & monitoring providers in the Terraform Registry
• Manage infrastructure and service monitoring
• Automate monitoring with the Terraform Datadog provider

External resources:

• Learn how to manage Grafana with Terraform

Database

Terraform can deploy, manage, and interact with database management systems. You can use cloud providers, like AWS, GCP, or Azure, to deploy database services such as RDS, and vendor providers to deploy databases like Cockroach, and MongoDB.

Terraform offers features beyond just deploying databases. It can manage database schemas, allowing you to version control database structures. You can also manage user permissions, providing consistent access controls across different environments. Terraform can also handle cross-database operations like creating read replicas or configuring cross-database access. It can also manage integrations of your database with other services, such as communication with VMs.

Upon database creation, you can also data seed your database, which is great for creating test environments that DBAs can quickly use. With IaC, you can make sure you're seeding the same data across multiple environments, which will increase the reliability of your tests, as you know your data will be consistent.

Benefits of using Terraform:

• Automate seed and migrate data during database deployment
• Provision and configure database instances
• Manage cross-database and service integrations
• Automate backup plans in multi-cloud/region

HashiCorp resources:

• List of database providers in the Terraform Registry

Version control systems

Version control systems (VCS), such as GitHub, GitLab, and Bitbucket, allow you to store and version your code. VCS fosters collaboration by making it easy to share and collaborate on code with peers. Terraform can manage the installation and configuration of on-prem VSC systems and configure VCS as a service.

Automating VCS configuration with Terraform improves the reliability and security of your infrastructure by applying the same controls you use for managing applications and infrastructure. By managing your VCS as code, you can reduce manual configuration errors, integrate VCS configuration into CI/CD, and enforce security best practices at a granular level.

Benefits of using Terraform:

• Set default branches and branch protections
• Create branches and repositories
• Manage code releases
• Create and manage security controls, users, and groups
• Install and configure your VCS in the cloud and on-prem

HashiCorp resources:

• List of version control systems in the Terraform Registry
• Manage GitHub Users, Teams, and Repository Permissions

Security and authentication

Terraform can manage cloud native security along with vendor security tools. You can deploy identity and access management, encryption keys, network security like firewalls, and auditing. You can manage all these controls as code, allowing you to version and audit code changes.

Terraform can also connect to secret management tools like HashiCorp Vault to use its security features. You will have more control over your secrets and can select a secret management solution that provides value to your specific organizational needs.

Benefits of using Terraform:

• Manage your cloud IAM
• Create your security and auth with IaC
• Integrate third-party security services into your cloud environment Audit your security changes though VCS
• Connect to secret management tools like Vault

HashiCorp resources:

• List of security and authentication providers in the Terraform Registry

Next steps

You can use Terraform to codify your entire infrastructure stack with a variety of providers in the Terraform Registry. When you use Terraform providers, you are able to use infrastructure as code, gaining benefits like reducing human error, version control, and automation.

To learn more about how to use Terraform and HCL, visit our Terraform code style guide.