New in Terraform: Manage global secondary index drift in Amazon DynamoDB

If you’ve ever adjusted Amazon DynamoDB global secondary index capacity (GSI) outside Terraform, you know how Terraform detects drift and forces unwanted reverts. With Terraform’s new aws_dynamodb_global_secondary_index resource, you can address this problem.

The new aws_dynamodb_global_secondary_index resource treats each GSI as an independent resource with its own lifecycle management. You can use this feature to make capacity adjustments for GSI and tables outside of Terraform.

In this post, I demonstrate how to use Terraform’s new aws_dynamodb_global_secondary_index resource to manage GSI drift selectively. I walk you through the limitations of current approaches and guide you through implementing the solution.

The problem: Terraform drift and GSI management

Before diving into the solution, let’s establish what drift means in infrastructure management. In infrastructure as code (IaC), drift occurs when the actual state of your infrastructure differs from what’s defined in your Terraform configuration. Terraform detects drift by comparing the desired state (your .tf configuration files), the last known state (stored in terraform.tfstate), and the actual state (queried from AWS). When these don’t match, Terraform reports drift and proposes changes to reconcile the difference.

DynamoDB GSIs often require capacity adjustments for various operational reasons: load testing, capacity planning, emergency performance requirements, or managing warm throughput. Your DynamoDB capacity can also be changed by autoscaling events. Whenever you make these changes outside of Terraform, it creates drift between Terraform’s configuration and AWS reality.

For example, let’s assume your analytics team runs a daily report that queries a GSI heavily. The report runs at 2:00 AM and needs 50 read capacity units (RCUs), but during normal hours, 5 RCUs is sufficient. Your operations team manually increases capacity before the report runs to handle the load.

At 1:50 AM, your ops team increases capacity from 5 to 50 using AWS Command Line Interface (AWS CLI). The report runs from 2:00 AM to 3:00 AM with the higher capacity. Later that day, when you run terraform plan to deploy an unrelated change, Terraform detects drift because the actual capacity (50) doesn’t match your configuration (5). Terraform wants to revert the capacity back to 5, which would interfere with your operational capacity management.

The common workaround and its limitations

A common workaround is to use ignore_changes = [global_secondary_index] in your table’s lifecycle block. This prevents Terraform from detecting capacity drift. However, this approach is too broad—it ignores all GSI changes, not just capacity. Because global_secondary_index is a complex nested type, ignore_changes only works at the top-level, not at individual attributes. If someone accidentally deletes a GSI or modifies its key schema, Terraform won’t detect it. You can’t distinguish between intentional capacity tuning and accidental GSI deletions.

The solution: Separate GSI resources

The new aws_dynamodb_global_secondary_index resource treats each GSI as an independent resource with its own lifecycle management. This gives you granular control over which attributes to ignore for each GSI while still detecting important changes like deletions or schema modifications.

Prerequisites

Before you begin, verify you have:

• An AWS account with permissions to create and manage DynamoDB tables (needed to create the test resources that you will use in examples)
• An Amazon Elastic Compute Cloud (Amazon EC2) instance running Amazon Linux with an AWS Identity and Access Management (IAM) role that has DynamoDB permissions
• AWS Command Line Interface (AWS CLI) installed and configured
• AWS Terraform Provider version 6.28.0 or later (the aws_dynamodb_global_secondary_index resource was introduced in v6.28.0)

The aws_dynamodb_global_secondary_index resource is currently marked as experimental in the Terraform AWS provider. This means the schema or behavior might change without notice, and isn’t subject to the backwards compatibility guarantee of the provider.

You must set the environment variable TF_AWS_EXPERIMENT_dynamodb_global_secondary_index to enable this experimental resource. Without this environment variable, Terraform will return an error when attempting to use aws_dynamodb_global_secondary_index. Set it before running any Terraform commands:

export TF_AWS_EXPERIMENT_dynamodb_global_secondary_index=1

Test thoroughly in non-production environments before using in production. You are welcome to provide feedback at GitHub Issue #45640.

If you’re upgrading from AWS Provider v5.x to v6.x, review the v6.0.0 upgrade guide for breaking changes before proceeding.

Install Terraform on Amazon Linux:

# Update system
sudo yum update -y

# Install yum-config-manager
sudo yum install -y yum-utils

# Add HashiCorp repository
sudo yum-config-manager --add-repo https://rpm.releases.hashicorp.com/AmazonLinux/hashicorp.repo

# Install Terraform
sudo yum -y install terraform

# Verify installation
terraform --version

Using the new resource

Create a provisioned capacity table with two GSIs using the new separate resource method. You’ll create main.tf where the table and GSIs are defined as independent resources.

Table and GSI keys:

Configuration:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.28"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

# DynamoDB Table without GSI blocks (GSIs managed separately)
# Only define attributes that are used as table keys (hash_key/range_key)
# GSI attributes are defined in the separate aws_dynamodb_global_secondary_index resources
resource "aws_dynamodb_table" "test_table" {
  name           = "GSITestTable"
  billing_mode   = "PROVISIONED"
  read_capacity  = 5
  write_capacity = 5
  hash_key       = "id"
  range_key      = "timestamp"

  attribute {
    name = "id"
    type = "S"
  }

  attribute {
    name = "timestamp"
    type = "N"
  }

  tags = {
    Name        = "GSITestTable"
    Environment = "test"
    Purpose     = "Testing new GSI resource"
  }
}

# GSI as a separate resource
resource "aws_dynamodb_global_secondary_index" "status_index" {
  table_name = aws_dynamodb_table.test_table.name
  index_name = "StatusUserIndex"

  # Provisioned throughput configuration
  provisioned_throughput {
    read_capacity_units  = 5
    write_capacity_units = 5
  }

  # key_schema now includes attribute_type (required in new resource)
  key_schema {
    attribute_name = "status"
    attribute_type = "S"
    key_type       = "HASH"
  }

  key_schema {
    attribute_name = "user_id"
    attribute_type = "S"
    key_type       = "RANGE"
  }

  # Projection configuration
  projection {
    projection_type = "ALL"
  }

  # With the new separate resource, you can now ignore specific attributes per GSI
  lifecycle {
    ignore_changes = [provisioned_throughput]
  }
}

# Second GSI to test multiple independent GSIs
resource "aws_dynamodb_global_secondary_index" "timestamp_index" {
  table_name = aws_dynamodb_table.test_table.name
  index_name = "TimestampIndex"

  # Provisioned throughput configuration
  provisioned_throughput {
    read_capacity_units  = 3
    write_capacity_units = 3
  }

  key_schema {
    attribute_name = "timestamp"
    attribute_type = "N"
    key_type       = "HASH"
  }

  # Projection configuration
  projection {
    projection_type = "KEYS_ONLY"
  }

  # This GSI is fully managed by Terraform (no ignore_changes)
}

Deploy the resources:

# Set the required environment variable
export TF_AWS_EXPERIMENT_dynamodb_global_secondary_index=1

terraform init
terraform plan
terraform apply

Test selective ignore_changes by manually changing the capacity of StatusUserIndex (the one with ignore_changes):

aws dynamodb update-table \
  --table-name GSITestTable \
  --region us-east-1 \
  --global-secondary-index-updates '[{
    "Update": {
      "IndexName": "StatusUserIndex",
      "ProvisionedThroughput": {
        "ReadCapacityUnits": 10,
        "WriteCapacityUnits": 10
      }
    }
  }]'

# Wait for update to complete
sleep 30

Running terraform plan shows No changes even though StatusUserIndex capacity changed to 10/10 in AWS. This occurs because of ignore_changes = [provisioned_throughput].

Verify drift detection still works by manually changing TimestampIndex (the one without ignore_changes):

aws dynamodb update-table \
  --table-name GSITestTable \
  --region us-east-1 \
  --global-secondary-index-updates '[{
    "Update": {
      "IndexName": "TimestampIndex",
      "ProvisionedThroughput": {
        "ReadCapacityUnits": 8,
        "WriteCapacityUnits": 8
      }
    }
  }]'

# Wait for update to complete
sleep 30

Running terraform plan detects the drift and proposes to change TimestampIndex capacity from 8 back to 3. This demonstrates that:

• StatusUserIndex shows no changes (capacity ignored as intended)
• TimestampIndex shows drift detection (capacity changes detected)
• Each GSI has independent lifecycle management
• You can selectively ignore specific attributes per GSI
• Terraform still detects important changes on GSIs without ignore_changes.

The key differences from the traditional method are that the table defines attributes used by the table itself (id, timestamp), while GSI-specific attributes (status, user_id) are defined in the separate GSI resource’s key_schema blocks with their attribute_type (required in new resource). If a GSI reuses a table attribute, that attribute remains in the table’s attribute block. The GSI is a separate resource with its own lifecycle.

Benefits of the new resource

The new resource model provides several advantages. You can now ignore specific attributes of a GSI without affecting other GSIs and automated scripts can adjust capacity based on traffic patterns without creating Terraform drift. You still track important changes like key schema modifications, confirming no accidental GSI deletions or reconfigurations. Terraform state remains the source of truth for GSI structure, while DynamoDB APIs show actual runtime capacity.

Each GSI can have its own lifecycle rules, providing independent management. The new resource model follows Terraform best practices where each resource manages one logical infrastructure component, dependencies are explicit through resource references, and state management is more straightforward.

The new resource fully supports warm throughput configuration for on-demand tables. Warm throughput is a DynamoDB capability that you can use to specify baseline capacity for on-demand tables, helping you manage performance and costs more predictably. This is how you can test it.

Create ondemand.tf:

resource "aws_dynamodb_table" "ondemand_test" {
  name         = "OnDemandGSITest"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "id"

  attribute {
    name = "id"
    type = "S"
  }
}

resource "aws_dynamodb_global_secondary_index" "category_index" {
  table_name = aws_dynamodb_table.ondemand_test.name
  index_name = "CategoryIndex"

  key_schema {
    attribute_name = "category"
    attribute_type = "S"
    key_type       = "HASH"
  }

  # Projection configuration
  projection {
    projection_type = "ALL"
  }

  # Warm throughput configuration (attribute, not a block)
  warm_throughput = {
    read_units_per_second  = 13000
    write_units_per_second = 5000
  }

  lifecycle {
    # Allow manual warm throughput tuning
    ignore_changes = [warm_throughput]
  }
}

Deploy and test:

# Set the required environment variable if not already set
export TF_AWS_EXPERIMENT_dynamodb_global_secondary_index=1

terraform apply

# Change warm throughput manually
aws dynamodb update-table \
  --table-name OnDemandGSITest \
  --region us-east-1 \
  --global-secondary-index-updates '[{
    "Update": {
      "IndexName": "CategoryIndex",
      "WarmThroughput": {
        "ReadUnitsPerSecond": 14000,
        "WriteUnitsPerSecond": 5100
      }
    }
  }]'

# Wait for update
sleep 30

# Run terraform plan
terraform plan

Terraform shows No changes because warm throughput changes are ignored as expected.

Before moving to the next section, destroy the on-demand test resources:terraform destroy

Migration example

Now that you’ve seen how the new resource works, let’s walk through a complete hands-on migration of existing infrastructure. Start with a table using the traditional nested GSI approach, then migrate it to the new separate resource method without any downtime.

Step 1: Create infrastructure with traditional method

Create a DynamoDB table with a GSI using the traditional nested block approach.

Create a file called migration-old.tf:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.28"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

# Traditional approach: GSI defined as nested block
resource "aws_dynamodb_table" "products" {
  name           = "ProductsTable"
  billing_mode   = "PROVISIONED"
  read_capacity  = 5
  write_capacity = 5
  hash_key       = "ProductId"

  attribute {
    name = "ProductId"
    type = "S"
  }

  attribute {
    name = "Category"
    type = "S"
  }

  # GSI defined as nested block (TRADITIONAL METHOD)
  global_secondary_index {
    name               = "CategoryIndex"
    hash_key           = "Category"
    projection_type    = "ALL"
    read_capacity      = 3
    write_capacity     = 3
  }

  tags = {
    Name        = "ProductsTable"
    Environment = "migration-demo"
  }
}

Deploy this infrastructure:

terraform init
terraform plan
terraform apply

Verify the table and GSI were created:

aws dynamodb describe-table --table-name ProductsTable --region us-east-1 \
  --query 'Table.GlobalSecondaryIndexes[0].IndexName'

Output:

CategoryIndex

Step 2: Prepare for migration

Before migrating, backup your Terraform state:

terraform state pull > backup-before-migration.tfstate

Set the required environment variable:

export TF_AWS_EXPERIMENT_dynamodb_global_secondary_index=1

Step 3: Update your Terraform configuration

Create a new file called migration-new.tf with the updated configuration. Keep both files for now—you will remove the old one after import.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.28"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

# Updated table: GSI block removed
resource "aws_dynamodb_table" "products" {
  name           = "ProductsTable"
  billing_mode   = "PROVISIONED"
  read_capacity  = 5
  write_capacity = 5
  hash_key       = "ProductId"

  # Only define attributes used by the table's own keys
  attribute {
    name = "ProductId"
    type = "S"
  }

  tags = {
    Name        = "ProductsTable"
    Environment = "migration-demo"
  }
}

# NEW: GSI as a separate resource
resource "aws_dynamodb_global_secondary_index" "category_index" {
  table_name = aws_dynamodb_table.products.name
  index_name = "CategoryIndex"

  # Provisioned throughput configuration
  provisioned_throughput {
    read_capacity_units  = 3
    write_capacity_units = 3
  }

  key_schema {
    attribute_name = "Category"
    attribute_type = "S"
    key_type       = "HASH"
  }

  # Projection configuration
  projection {
    projection_type = "ALL"
  }

  # Allow ops team to adjust capacity without Terraform reverting it
  lifecycle {
    ignore_changes = [provisioned_throughput]
  }
}

Step 4: Remove the old configuration

Now remove or rename the old file:

mv migration-old.tf migration-old.tf.backup

At this point, if you run terraform plan, you’ll see that Terraform wants to remove the GSI from the table (because the nested block is gone) and create a new separate GSI resource.

Don’t apply yet. This would cause downtime. Instead, import the existing GSI.

Step 5: Import the existing GSI

Import the existing GSI into the new resource’s state:

# Import format: 'table_name,index_name'
terraform import aws_dynamodb_global_secondary_index.category_index \
  'ProductsTable,CategoryIndex'

Output:

aws_dynamodb_global_secondary_index.category_index: Importing from ID "ProductsTable,CategoryIndex"...
aws_dynamodb_global_secondary_index.category_index: Import prepared!
  Prepared aws_dynamodb_global_secondary_index for import
aws_dynamodb_global_secondary_index.category_index: Refreshing state... [id=ProductsTable,CategoryIndex]

Import successful!

Step 6: Verify the migration

Run terraform plan to verify:

terraform plan

Expected output:

aws_dynamodb_table.products: Refreshing state... [id=ProductsTable]
aws_dynamodb_global_secondary_index.category_index: Refreshing state... [id=ProductsTable,CategoryIndex]

No changes. Your infrastructure matches the configuration.

If you see No changes, the migration was successful. The GSI is now managed as a separate resource.

Migration summary

To complete a migration, you started with a traditional nested GSI configuration, which you then migrated to separate GSI resources without downtime using terraform import. You then verified the migration with terraform plan showing No changes, after which you successfully transitioned to the new resource model.

Key takeaways:

• Migration uses terraform import
• No AWS resources are modified or recreated
• The GSI continues to exist throughout the migration with zero downtime
• After migration, you have granular control over what to ignore with ignore_changes
• The migration process is safe and reversible

Migration considerations

Do not combine aws_dynamodb_global_secondary_index resources with global_secondary_index blocks on aws_dynamodb_table. Doing so might cause conflicts, perpetual differences, and GSIs being overwritten.

When migrating, follow these steps:

1. Backup state: terraform state pull > backup.tfstate
2. Set environment variable: export TF_AWS_EXPERIMENT_dynamodb_global_secondary_index=1
3. Update configuration: Remove the GSI block from table the table and create a new GSI resource
4. Import existing GSI: terraform import <resource> 'table_name,index_name'
5. Verify: Run terraform plan, it should show No changes
6. Test: Manually change capacity and verify that Terraform ignores the change

You won’t experience downtime during migration if done correctly using terraform import. The GSI continues to exist in AWS throughout the migration. The terraform import command only updates Terraform’s state file—it doesn’t modify AWS resources.

If your table has multiple GSIs, migrate them one at a time:

1. Import the first GSI and verify with terraform plan
2. Import the second GSI and verify with terraform plan
3. Continue until all GSIs are migrated

This reduces risk and simplifies troubleshooting.

Comparison: Traditional compared to new method

The following table summarizes the key differences between the traditional nested block approach and the new separate resource method:

Clean up

To avoid incurring future charges, delete the resources you created in this walkthrough:

# Destroy all Terraform-managed resources
terraform destroy

# Confirm the deletion when prompted
# Type 'yes' to proceed

If you created any resources manually during testing, make sure to delete those as well through the AWS Management Console or AWS CLI to avoid incurring future costs.

Conclusion

In this post, I showed you how the new aws_dynamodb_global_secondary_index resource solves the long-standing challenge of managing DynamoDB GSI drift in Terraform. The all-or-nothing nature of ignoring nested global_secondary_index blocks created a gap between operational flexibility and infrastructure governance.

By treating GSIs as first-class resources, you gain granular control with selective ignore_changes for specific GSI attributes, independent management where each GSI has its own lifecycle rules, better drift detection that tracks important changes while allowing operational adjustments, and a more straightforward architecture with separation of concerns between table and index configuration.

Remember that the aws_dynamodb_global_secondary_index resource is currently marked as experimental. While it provides powerful capabilities for managing GSI drift, be aware that:

• The schema or behavior might change in future provider versions
• You must set the environment variable TF_AWS_EXPERIMENT_dynamodb_global_secondary_index=1 to enable this resource
• It’s not subject to the backwards compatibility guarantee of the provider
• You can’t mix this resource with traditional global_secondary_index blocks on the same table

Always test thoroughly in non-production environments and monitor provider release notes for updates. If you have feedback, provide it at GitHub Issue #45640 to help shape the future of this feature.

About the authors