Terraform is amazing for spinning up infrastructure fast, but what happens when your small project grows into a full-blown production system?
At first, your main.tf file might look simple and clean. But over time, things start to get messy.
- Too many resources in one file.
- Hardcoded values everywhere.
- Scaling across multiple environments becomes a nightmare.
Sounds familiar? You’re not alone!
In this post, we’ll explore how to evolve Terraform configurations over time, making them:
- Modular – Reusable and organized.
- Scalable – Easy to manage across environments.
- Maintainable – So future-you won’t hate past-you.
Let’s level up your Terraform game!
1. The Evolution of a Terraform Configuration
Terraform setups usually follow this evolution path:
1. The “Just Make It Work” Stage – A single main.tf file with everything inside.
2. The “Oops, This Is a Mess” Stage – Multiple .tf files, but still unorganized.
3. The “We Need Structure” Stage – Breaking out reusable modules.
4. The “Full Automation” Stage – Remote state, workspaces, and pipelines.
2. Breaking Up Large Terraform Files
A common Terraform mistake? Jamming everything into one massive file.
Example of a Messy Terraform Setup (main.tf contains everything)
provider "aws" {
region = "us-east-1"
}
resource "aws_instance" "web" {
ami = "ami-123456"
instance_type = "t2.micro"
}
resource "aws_s3_bucket" "logs" {
bucket = "my-logs-bucket"
}
resource "aws_vpc" "main" {
cidr_block = "10.0.0.0/16"
}
Why This is Bad
- Hard to read and maintain.
- Resources aren’t logically grouped.
- Impossible to reuse configurations.
The Fix: Split Into Logical Files
/terraform ├── main.tf # Calls modules ├── providers.tf # Provider configurations ├── variables.tf # Input variables ├── outputs.tf # Output values ├── vpc.tf # VPC resources ├── instances.tf # EC2 instances ├── s3.tf # S3 Buckets
Now, each file serves a clear purpose, making Terraform configurations easier to maintain.
3. Using Terraform Modules to Reuse Code
Once your infrastructure grows, you’ll find yourself copy-pasting code between environments. STOP!
Terraform modules solve this by allowing you to reuse configurations.
Example: Creating a Reusable EC2 Module
Folder Structure
/terraform ├── modules │ ├── ec2 │ │ ├── main.tf │ │ ├── variables.tf │ │ ├── outputs.tf ├── dev │ ├── main.tf ├── prod │ ├── main.tf
modules/ec2/main.tf (EC2 Module Code)
resource "aws_instance" "this" {
ami = var.ami
instance_type = var.instance_type
}
output "public_ip" {
value = aws_instance.this.public_ip
}
modules/ec2/variables.tf (Define Variables)
variable "ami" {}
variable "instance_type" {}
Using the Module in dev/main.tf
module "dev_ec2" {
source = "../modules/ec2"
ami = "ami-123456"
instance_type = "t2.micro"
}
Now, the same module can be used for prod, staging, etc.
4. Managing Multiple Environments with Workspaces
When you need multiple environments (dev, staging, prod), Terraform workspaces help avoid duplication.
Step 1: Create Workspaces
terraform workspace new dev terraform workspace new prod
Step 2: Reference Workspace in Terraform
variable "environment" {}
resource "aws_instance" "web" {
ami = "ami-123456"
instance_type = var.environment == "prod" ? "t3.large" : "t3.micro"
}
Now, switching environments is easy!
bashCopyEditterraform workspace select dev
terraform apply
5. Using Remote State for Collaboration
If multiple people work on Terraform, local state files (terraform.tfstate) don’t cut it.
The Fix: Use Remote State
terraform {
backend "s3" {
bucket = "my-terraform-state"
key = "global/terraform.tfstate"
region = "us-east-1"
}
}
Now, state is stored in S3, making Terraform collaborative and safe!
6. Automating Terraform with CI/CD
Once Terraform is modular and structured, automate everything!
Example: GitHub Actions for Terraform Automation
name: Terraform CI/CD
on:
push:
branches:
- main
jobs:
terraform:
runs-on: ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout@v2
- name: Setup Terraform
uses: hashicorp/setup-terraform@v1
- name: Terraform Init
run: terraform init
- name: Terraform Plan
run: terraform plan
- name: Terraform Apply
run: terraform apply -auto-approve
Now, Terraform runs automatically on every commit!
Terraform Evolution Cheat Sheet
| Stage | Changes |
|---|---|
| Basic Terraform | A single main.tf file. |
| Organized Terraform | Split resources into multiple .tf files. |
| Modular Terraform | Reuse components using modules. |
| Multiple Environments | Use workspaces to manage dev, prod, etc. |
| Remote State | Store Terraform state in S3/Azure/GCS for collaboration. |
| Automated Terraform | Run Terraform in CI/CD pipelines. |
Follow this path, and your Terraform setup will be rock solid!
Wrapping Up
Terraform setups start small but grow over time—and without structure, things get out of control fast.
Quick Recap:
- Break up Terraform files for better organization.
- Use modules to reuse and simplify Terraform code.
- Manage environments with Terraform workspaces.
- Use remote state for better collaboration.
- Automate Terraform with CI/CD pipelines.
Now, go refactor your Terraform setup and future-proof it!
What’s Next?
What happens when things go wrong? In the next post, “Automating Disaster Recovery with Terraform,” we’ll cover how to build self-healing infrastructure, backup state files, and ensure quick recovery from failures.