Terraform - State

Terraform must store state about your managed infrastructure and configuration. This state is used by Terraform to map real world resources to your configuration, keep track of metadata, and to improve performance for large infrastructures.

terraform.tfstate, using simple JSON format, when you run Terraform, it can fetch the latest status of services from AWS and compare that to what’s in your Terraform configurations to determine what changes need to be applied.

Shared Storage for State Files

By default, Terraform stores in local. Because this file must exist, it makes working with Terraform in a team complicated since it is a frequent source of merge conflicts. Remote state helps alleviate these issues.
After version 0.9, terraform introduce “backends” to replace “remote state”, Click here for detail

1. Create a S3 bucket

   provider "aws" {
   	region = "ap-northeast-1"
   }
   
   # S3
   resource "aws_s3_bucket" "terraform_state" {
   	bucket = "yongfeiuall_terraform_state"
   	versioning {
   		enabled = true
   	}
   	lifecycle {
   		prevent_destroy = true
   	}
   }

2. configure store state to S3
   a. add this in terraform file

   terraform {
     backend "s3" {
       bucket = "yongfeiuall-terraform-state" 
       key = "global/s3/terraform.tfstate" 
   	region = "ap-northeast-1"
   	encrypt = "true"
     }
   }

   b. execute terraform init

   D:\terraform\Example\stateS3>terraform init
   
   Initializing the backend...
   Do you want to copy existing state to the new backend?
     Pre-existing state was found while migrating the previous "local" backend to the
     newly configured "s3" backend. No existing state was found in the newly
     configured "s3" backend. Do you want to copy this state to the new "s3"
     backend? Enter "yes" to copy and "no" to start with an empty state.
   
     Enter a value: yes
   
   
   Successfully configured the backend "s3"! Terraform will automatically
   use this backend unless the backend configuration changes.
   
   Initializing provider plugins...
   
   The following providers do not have any version constraints in configuration,
   so the latest version was installed.
   
   To prevent automatic upgrades to new major versions that may contain breaking
   changes, it is recommended to add version = "..." constraints to the
   corresponding provider blocks in configuration, with the constraint strings
   suggested below.
   
   * provider.aws: version = "~> 1.15"
   
   Terraform has been successfully initialized!
   
   You may now begin working with Terraform. Try running "terraform plan" to see
   any changes that are required for your infrastructure. All Terraform commands
   should now work.
   
   If you ever set or change modules or backend configuration for Terraform,
   rerun this command to reinitialize your working directory. If you forget, other
   commands will detect it and remind you to do so if necessary.

With remote state enabled, Terraform will automatically pull the latest state from
this S3 bucket before running a command, and automatically push the latest state
to the S3 bucket after running a command.
E.g., add the below info in terraform file, and run terraform apply again.

output "s3_bucket_arn" {
value = "${aws_s3_bucket.terraform_state.arn}"
}

Locking state files

If the state file is stored remotely so that many people can access it, then you risk multiple people attempting to make changes to the same file at the exact same time. So we need to provide a mechanism that will “lock” the state if its currently in-use by another user. We can accomplish this by creating a dynamoDB table for terraform to use.

1. Create the dynamoDB table

   provider "aws" {
   	region = "ap-northeast-1"
   }
   
   # create a dynamodb table for locking the state file
   resource "aws_dynamodb_table" "dynamodb-terraform-state-lock" {
     name = "yongfeiualll-terraform-state-lock"
     hash_key = "LockID"
     read_capacity = 20
     write_capacity = 20
    
     attribute {
       name = "LockID"
       type = "S"
     }
    
     tags {
       Name = "DynamoDB Terraform State Lock Table"
     }
   }

Run terraform apply

1. Modify the Terraform S3 backend resource

   terraform {
     backend "s3" {
       bucket = "yongfeiuall-terraform-lock" 
   	dynamodb_table = "yongfeiualll-terraform-state-lock"
       key = "global/s3/terraform.tfstate" 
   	region = "ap-northeast-1"
   	encrypt = "true"
     }
   }
   
   provider "aws" {
   	region = "ap-northeast-1"
   }
   
   # S3
   resource "aws_s3_bucket" "terraform_lock" {
   	bucket = "yongfeiuall-terraform-lock"
   	versioning {
   		enabled = true
   	}
   	lifecycle {
   		prevent_destroy = true
   	}
   }

2. Run terraform init, terraform plan and terraform apply commands

Isolating state files

Recommend using separate Terraform folders (and therefore separate state files) for each environment, and within each environment, each “component.”
Here is the file layout for our typical Terraform project:

stage
  └ vpc
  └ services
      └ frontend-app
      └ backend-app
          └ variables.tf
          └ outputs.tf
          └ main.tf
  └ data-storage
      └ mysql
      └ redis
mgmt
  └ vpc
  └ services
      └ bastion-host
      └ jenkins
global
  └ iam
  └ route53

• stage: an environment for non-production workloads
• mgmt: an environment for DevOps tooling (e.g. bastion host, Jenkins).
• global: a place to put resources that are used across all environments, such as user management (IAM in AWS) and DNS management (Route53 in AWS).

Within each environment, we have separate folders for each “component.”

Within each component, we have the actual Terraform templates, which we organize according to the following naming conventions:

• variables.tf: input variables.
• outputs.tf: output variables.
• main.tf: the actual resources.