Terraform Remote State Storage with S3 and Azure Blob

Local state files — stored on your laptop — break as soon as a second person joins your team. Two engineers running Terraform at the same time with separate local state files will overwrite each other's state, create duplicate resources, or miss each other's changes entirely. Remote state solves this by storing the state file in a shared, durable, access-controlled location.

What Remote State Does

Instead of saving terraform.tfstate to your local disk, Terraform saves it to a remote storage backend. Every team member's Terraform reads from and writes to the same file in that shared location.

Diagram: Local State vs Remote State

LOCAL STATE (broken for teams):
  Engineer A laptop: terraform.tfstate (version 5)
  Engineer B laptop: terraform.tfstate (version 3)  <-- out of sync!
  Both apply at the same time → CONFLICT → corrupted infrastructure

REMOTE STATE (correct for teams):
  S3 bucket: terraform.tfstate (always the latest version)
       ^
       |
  Engineer A --------reads/writes-------> S3
  Engineer B --------reads/writes-------> S3
  CI/CD pipeline ---reads/writes-------> S3

Backend Configuration

You configure where Terraform stores state using a backend block inside the terraform block. Backends are configured before resources — they tell Terraform where to find and save its state.

Using AWS S3 as a Backend

S3 is the most widely used Terraform backend for AWS-based teams. You store state in an S3 bucket and use a DynamoDB table for state locking.

Step 1: Create the S3 Bucket and DynamoDB Table (one time only)

These resources must exist before you configure the backend. Create them manually or with a separate Terraform configuration that uses local state.

# One-time bootstrap configuration (stored separately)
resource "aws_s3_bucket" "terraform_state" {
  bucket = "my-company-terraform-state"

  lifecycle {
    prevent_destroy = true
  }
}

resource "aws_s3_bucket_versioning" "state" {
  bucket = aws_s3_bucket.terraform_state.id
  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "state" {
  bucket = aws_s3_bucket.terraform_state.id
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

resource "aws_dynamodb_table" "state_lock" {
  name         = "terraform-state-lock"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }
}

Step 2: Configure the Backend

terraform {
  backend "s3" {
    bucket         = "my-company-terraform-state"
    key            = "projects/webapp/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "terraform-state-lock"
  }
}

After adding this block, run terraform init. Terraform detects the backend change, asks if you want to migrate existing local state to S3, and moves the state file automatically.

The key Argument: Organising Multiple Projects

The key argument is the path within the S3 bucket where the state file is stored. Use a consistent naming convention to organise state files for multiple projects.

# Project 1 backend
key = "projects/webapp/terraform.tfstate"

# Project 2 backend
key = "projects/data-pipeline/terraform.tfstate"

# Environment-specific (within a project)
key = "projects/webapp/prod/terraform.tfstate"
key = "projects/webapp/staging/terraform.tfstate"

Using Azure Blob Storage as a Backend

For Azure-based teams, Azure Blob Storage provides the same capability as S3. Azure storage natively supports blob leases for state locking — no separate locking table is needed.

terraform {
  backend "azurerm" {
    resource_group_name  = "terraform-state-rg"
    storage_account_name = "mycompanytfstate"
    container_name       = "tfstate"
    key                  = "projects/webapp/prod.terraform.tfstate"
  }
}

Backend Block Cannot Use Variables

A common surprise: the backend block does not support variable references or expressions. Every value must be a literal string.

# WRONG — this fails
terraform {
  backend "s3" {
    bucket = var.state_bucket   # Error: variables not allowed here
  }
}

# CORRECT — use literal values
terraform {
  backend "s3" {
    bucket = "my-company-terraform-state"
  }
}

To handle different values per environment, use partial backend configuration with the -backend-config flag at init time:

# backend.hcl file
bucket         = "my-company-terraform-state"
key            = "webapp/prod/terraform.tfstate"
region         = "us-east-1"
dynamodb_table = "terraform-state-lock"

# Then run:
terraform init -backend-config=backend.hcl

Key Points

• Local state breaks for teams — remote state stores the state file in a shared, durable location like S3 or Azure Blob.
• Configure remote state with a backend block inside the terraform block and run terraform init to migrate.
• For S3 backends, pair the bucket with a DynamoDB table for state locking to prevent concurrent applies.
• Use the key argument to organise multiple projects and environments within the same storage bucket.
• The backend block does not support variables — use literal values or pass a -backend-config file at init time.