Over time, Terraform code can become complex. Refactoring, similar to spring cleaning, brings order and clarity. In this article, we’ll explore how to refactor your Terraform code, covering renaming resources, moving resources to modules, and even migrating resources between state files.

Why Refactor Your Terraform Code?

Before diving into the how-to, let’s shed light on the benefits of refactoring Terraform code:

• Enhanced Readability and Maintainability: Cleaner code with logical structuring improves understanding and simplifies future modifications.
• Reduced Complexity: Breaking down large configurations into smaller, modular units fosters manageability and prevents tangled code.
• Improved Reusability: Extracting reusable code into modules promotes efficiency and consistency across projects.
• Streamlined Collaboration: Well-organised code enables smoother collaboration and knowledge sharing within teams.

Resist the Urge to Edit State Files Manually

Tempting as it may be to modify the Terraform state file directly, resist the urge! Manual edits can introduce inconsistencies and conflicts, which puts your infrastructure at risk. Even minor changes can snowball into bigger problems. Fortunately, Terraform provides built-in tools for safe and reliable refactoring:

The ‘moved’ Block for Renaming and Intra-State Migrations

• This method offers a convenient way to rename resources or move them within the same state file.
• Simply add a ‘moved’ block to your Terraform configuration, specifying the original and new locations of the resource.

While straightforward, this method has limitations:

• Scalability: Requires a separate ‘moved’ block for each resource, becoming tedious for large-scale refactoring.
• Limited Scope: Only supports moving resources within the same state file.

Terraform state mv for Advanced Refactoring and Inter-State Migrations

• This command-line tool provides greater flexibility for complex refactoring scenarios.
• You can use it to rename resources, move them between state files, and even handle different object types.

While more powerful, this method requires careful execution and understanding of state file management:

• Manual Scripting: Scripting is often necessary for batch operations, demanding more technical expertise.
• State Consistency: Ensure your state file accurately reflects the intended changes before using terraform state mv.

Refactoring Goals

The objective in Terraform refactoring is to adjust resources within the state file without the need to destroy or modify already deployed resources.

Let’s utilise the code below to gain a deeper understanding of both approaches.

resource "azurerm_resource_group" "rg-storage" {
  name     = var.rg_name
  location = "uksouth"
}
resource "azurerm_storage_account" "storage-account" {
  name                     = "tfrefactsa001"
  resource_group_name      = azurerm_resource_group.rg-storage.name
  location                 = azurerm_resource_group.rg-storage.location
  account_tier             = "Standard"
  account_replication_type = "GRS"
  tags = {
    environment = "production"
  }
}

With the resources above deployed, if we run the command terraform state list, the returned output might include the following entries:

As the infrastructure expands, we need to manage multiple storage accounts efficiently. In response, we decided to utilise a for_each meta-argument. Here's what it looks like:

locals{
  st_names = ["tfrefactsa001"]
}
resource "azurerm_resource_group" "rg-storage" {
  name     = var.rg_name
  location = "uksouth"
}
resource "azurerm_storage_account" "storage-account" {
  for_each                 = toset(local.st_names)
  name                     = each.value
  resource_group_name      = azurerm_resource_group.rg-storage.name
  location                 = azurerm_resource_group.rg-storage.location
  account_tier             = "Standard"
  account_replication_type = "GRS"
  tags = {
    environment = "production"
  }
}

Indeed, if we run terraform plan at this stage Terraform will attempt to destroy the current storage account and recreate it with the same name, using the updated resource structure. This occurs because Terraform views each instance of the azurerm_storage_account resource as an independent entity and does not recognise that the existing resource is simply being updated.

Let’s first examine the ‘moved’ block

moved {
  from = SOURCE
  to   = DESTINATION
}

In this block, the from attribute represents the original location of the resource, and the to attribute represents the new location where the resource has been moved or renamed. This block tells Terraform about the resource move, ensuring Terraform tracks changes accurately.

Below is the updated version of the provided code, including the ‘moved’ block:

locals{
  st_names = ["tfrefactsa001"]
}
resource "azurerm_resource_group" "rg-storage" {
  name     = var.rg_name
  location = "uksouth"
}
resource "azurerm_storage_account" "storage-account" {
  for_each = toset(local.st_names)
  name                     = each.value
  resource_group_name      = azurerm_resource_group.rg-storage.name
  location                 = azurerm_resource_group.rg-storage.location
  account_tier             = "Standard"
  account_replication_type = "GRS"
  tags = {
    environment = "production"
  }
}
moved {
  from = azurerm_storage_account.storage-account
  to   = azurerm_storage_account.storage-account["tfrefactsa001"]
}

This time, when running terraform plan, no alterations are detected in the environment, indicating that it is safe to proceed with our changes with terraform apply:

Once the changes are applied, the ‘moved’ block must be removed from our code.

Finally, if we run the command terraform state list, we can confirm the resources have been updated in our state file:

Move resources to a module with Terraform CLI

terraform state mv [options] SOURCE DESTINATION

Unlike the Terraform ‘moved’ block, the terraform state mv CLI command empowers us to automate the process into a script, rather than duplicating the block for each resource to be renamed. Additionally, it enables us to transfer resources between different state files, as we are going to cover later.

After some time, we concluded that converting this code into a module would be ideal. However, we had already utilised this version of our code and generated not just 2, but 5… wait… 10 storage accounts. Using the previous method, we would need to create 10 ‘moved’ blocks one for each storage account. This illustrates one of the drawbacks of the ‘moved’ block, as it involves a significant amount of manual work and is prone to errors. In this scenario, the terraform state mv CLI will provide us with the flexibility required to move all resources at once.

We’ve recently made enhancements to our codebase, integrating the new ‘storageAccount’ module. Additionally, to facilitate the migration of our resources to this module, we’ve developed a Bash script moveResources.sh. Below, you’ll find our updated file structure for reference:

Terraform_Refactoring
├── main.tf
├── modules
│   └── storageAccount
│       ├── main.tf
│       └── variables.tf
├── moveResources.sh
└── terraform.tfstate

Running the command terraform state list we have the list of resources we need to move to the module:

Below is the updated version of our main.tf file:

locals {
  st_names = [
    "tfrefactsa001", "tfrefactsa002", "tfrefactsa003", "tfrefactsa004",
    "tfrefactsa005", "tfrefactsa006", "tfrefactsa007", "tfrefactsa008",
    "tfrefactsa009", "tfrefactsa010"
  ]
}
resource "azurerm_resource_group" "rg-storage" {
  name     = var.rg_name
  location = "uksouth"
}
module "storageAccount" {
  source              = "./modules/storageAccount"
  for_each            = toset(local.st_names)
  name                = each.value
  resource_group_name = azurerm_resource_group.rg-storage.name
  location            = azurerm_resource_group.rg-storage.location
}
output "st_names" {
  value = local.st_names
}

Note: We added the output of the locals ‘st_names’ that will allow us to iterate though the storage account names with our Bash script.

#!/bin/bash
# Export locals as JSON
terraform output -json > output.json
# Extract and iterate over st_names in bash
st_names=$(jq -r '.st_names.value[]' output.json)
for name in $st_names; do
  terraform state mv \
  "azurerm_storage_account.storage-account[\"$name\"]" \
  "module.storageAccount.azurerm_storage_account.storage-account[\"$name\"]"
done
# Clean up
rm output.json

In this Bash script, we begin by exporting the Terraform outputs into a JSON file named output.json. Subsequently, we used the jqcommand-line tool to extract the names of storage accounts from this JSON file, iterating over each storage account name. Within the loop, we use the terraform state mv command to rename the corresponding resources in the Terraform state, relocating them from their original address (e.g., ‘azurerm_storage_account.storage-account[“$name”]’) to a new address (e.g., ‘module.storageAccount.azurerm_storage_account.storage-account[“$name”]’). Additionally, an option -dry-run is available but not included in the script: if set, it would test the viability of the resource move before execution. Finally, we ensure cleanliness by deleting the temporary ‘output.json’ file. Let’s move forward by running the script:

To confirm the resources have been moved correctly to the module we should run terraform state list:

Moving resources between state files

To enhance isolation, improve the security and foster collaboration, it has been decided to transfer these resources to a different state file. We are aware of one drawback of the ‘moved’ block, which is scalability. The second limitation is that it only moves resources within a state file. Therefore, the only remaining built-in tool to move the resources to a different state file is using terraform state mv.

We have relocated our Terraform module to a new Terraform project, and it is now time to transfer our resources to the new state file. We will build our script based on the one previously used, as it shares many similarities. Here is the updated version of our script:

#!/bin/bash
# Define paths to state files
source_state="./terraform.tfstate"
target_state="../Terraform_storageAccount/terraform.tfstate"
# Export locals as JSON
terraform output -json > output.json
# Move resource group
terraform state mv -state=$source_state -state-out=$target_state \
  "azurerm_resource_group.rg-storage" "azurerm_resource_group.rg-storage"
# Extract and iterate over st_names in bash
st_names=$(jq -r '.st_names.value[]' output.json)
for name in $st_names; do
  # echo "Moving the storage account $name to the module"
  terraform state mv -state=$source_state -state-out=$target_state \
  "module.storageAccount.azurerm_storage_account.storage-account[\"$name\"]" \
  "module.storageAccount.azurerm_storage_account.storage-account[\"$name\"]"
done
# Clean up
rm output.json

The main difference between the previous script and this is the usage of the options -state, where we define the path of the state file from which we are moving the resources, and -state-out, where we specify the state file to which the resources will be moved.

To verify that we have successfully moved the resources from the original location, we can execute the terraform state list command and should observe the resources listed. Similarly, this can be repeated in the new Terraform project, where we should also observe the presence of the resources:

Choosing Between ‘moved’ Block and terraform state mv

Both approaches have their advantages and disadvantages, depending on your use case and preferences. The table below outlines some factors to consider when choosing between them:

In summary, if the refactoring task is relatively simple and limited to renaming or relocating resources within the same state file, the “moved” block may be enough. However, for more complex refactoring operations or when moving resources between separate state files, the terraform state mv command offers greater flexibility and automation, but it requires more careful handling. Ultimately, the choice between the two approaches depends on the specific requirements and constraints of the refactoring project.