Generating Terraform providers with Terramate

In a repository of many root modules, provider configuration is the file most likely to be copied from the neighbouring stack: the version constraints, a provider block with its region and credentials, an alias for each extra account. Here a stack declares only which providers it needs, and one generator renders the rest.

The provider boilerplate problem

None of that configuration describes infrastructure, and all of it has to be right. The cost shows up on every change that crosses stacks: moving the AWS provider from ~> 5.0 to ~> 6.0 is one edit per root module, and a stack that was copied before the change keeps the old constraint until somebody notices.

The obvious fixes do not hold. Provider configuration belongs to the root module: a reusable module may declare required_providers, but a module that carries its own provider block cannot be called with count, for_each or depends_on. That rules out pushing the configuration down into a shared module.

Symlinking one providers.tf into every stack fails for a different reason: the file is never identical, since the region, the AWS profile and the set of aliases differ per stack. Templating it with a script works, and then the repository owns a template engine and the drift problem moves into it.

What is missing is a declaration: each stack states which providers it needs, and something else renders the HCL. That is what Terramate’s code generation does.

Declaring providers as globals

Part one covers how globals inherit down a directory tree. The other half of the feature is that a labelled globals block contributes one key to a map, so the providers a stack sees are assembled from every level above it, and where a file sits decides which stacks see its entry:

stacks/
├── imports/
│   ├── 001_providers.tm.hcl             the generator
│   └── aws/provider.tm.hcl              the aws provider defaults
└── aws/
    ├── aws.tm.hcl                       imports the generator
    └── acme-development/                environment, aws_config_profile
        └── eu-west-1/                   region_name, imports the provider
            └── acm/stack.tm.hcl         the stack, plus its own aliases

The AWS provider is declared once, at the region level, in terms of globals the leaf directories supply:

globals "terraform" "providers" "aws" {
  enabled = true
  source  = "hashicorp/aws"
  version = "~> 5.0"
  config = {
    region  = global.region_name
    profile = global.aws_config_profile
  }
}

A stack adds only what is specific to it. Here the acm stack needs a second AWS provider pointed at a shared-services account, which it declares as the labelled key aws.shared-services:

globals "terraform" "providers" "aws.shared-services" {
  enabled = true
  config = {
    region  = global.region_name
    profile = "acme-shared-services"
  }
}

The enabled flag is what makes this composable: a provider declared high in the tree can be switched off for one subtree without deleting the declaration. Three files, three levels, one map:

Each level declares what it knows; a stack sees the union of everything above it.

Generating the provider blocks

The generator is a single generate_hcl block in stacks/imports, imported by the root of each subtree. It has two inputs, the map above and a map of credential references covered in the next section, and it renders five kinds of block from them:

One generator, two inputs, five kinds of output; the module block’s outputs feed the provider blocks that need a token.

Terraform allows one version constraint per provider across the whole configuration, so an alias must produce a provider block and no required_providers entry. Keeping the alias in the key, aws.shared-services, is how the generator’s lets, its local values, tell the two apart. Functions prefixed tm_ are Terraform’s functions run by Terramate at generation time:

lets {
  required_providers = {
    for k, v in tm_try(global.terraform.providers, {}) :
    k => {
      source  = v.source
      version = v.version
    }
    if tm_try(v.enabled, true) && tm_length(tm_split(".", k)) == 1
  }
  providers = {
    for k, v in tm_try(global.terraform.providers, {}) :
    k => v
    if tm_try(v.enabled, true) && tm_length(tm_split(".", k)) == 1
  }
  providers_aliases = {
    for k, v in tm_try(global.terraform.providers, {}) :
    k => v
    if tm_try(v.enabled, true) && tm_length(tm_split(".", k)) == 2
  }
}

The content block renders the maps with tm_dynamic: a required_providers block from the first, one provider block per entry of the second, each carrying its own config merged with the credential references covered in the next section. Inside a tm_dynamic the current entry is named after the block label, here provider:

content {
  terraform {
    tm_dynamic "required_providers" {
      attributes = let.required_providers
    }
  }

  tm_dynamic "provider" {
    for_each = let.providers
    labels   = [provider.key]
    attributes = tm_merge(
      tm_try(provider.value.config, {}),
      {
        for k, v in tm_try(let.credentials[provider.key], {}) :
        k => tm_hcl_expression(v)
      },
    )
  }
}

Aliases come from the third map, with the label and the alias attribute taken from the two halves of the key:

tm_dynamic "provider" {
  for_each = let.providers_aliases
  labels   = [tm_split(".", provider.key)[0]]
  # ... the same attributes merge as above ...

  content {
    alias = tm_split(".", provider.key)[1]
  }
}

terramate generate writes the result next to each stack. For the acm stack, two globals files and one alias declaration produce this:

// TERRAMATE: GENERATED AUTOMATICALLY DO NOT EDIT

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}
provider "aws" {
  profile = "acme-development"
  region  = "eu-west-1"
}
provider "aws" {
  profile = "acme-shared-services"
  region  = "eu-west-1"
  alias   = "shared-services"
}

Wiring credentials into providers

Most SaaS providers need a token, and the token comes from the lookup module described in its own note. A generated attribute is rendered as a value, so a reference to that module’s output would land in the output file as a quoted string. The generator keeps the references as strings and converts them with tm_hcl_expression, which emits a string as verbatim HCL. References that become attributes are kept apart from those that belong inside a nested block, because a block needs a tm_dynamic of its own:

lets {
  secrets = "module.providers_credentials.secrets"

  credentials = {
    firehydrant = {
      api_key = "${let.secrets}.firehydrant.api_key"
    }
  }

  credential_blocks = {
    github = {
      app_auth = {
        id              = "${let.secrets}.github.app_id"
        installation_id = "${let.secrets}.github.app_installation_id"
        pem_file        = "base64decode(${let.secrets}.github.private_key_b64)"
      }
    }
  }
}

The content block above merges let.credentials into each provider’s attributes; a nested block is rendered inside the provider’s own content, conditioned on the provider that has one:

tm_dynamic "provider" {
  for_each = let.providers
  labels   = [provider.key]
  # ... the attributes merge as above ...

  content {
    tm_dynamic "app_auth" {
      condition = provider.key == "github"
      attributes = {
        for k, v in let.credential_blocks.github.app_auth :
        k => tm_hcl_expression(v)
      }
    }
  }
}

The module block itself is generated from the intersection of the providers a stack enables and the providers the two maps hold credentials for, and only when that intersection is not empty:

lets {
  # path from the stack back up to the repository root, e.g. ../../..
  modules = "${terramate.stack.path.to_root}/modules"

  credential_providers = tm_setintersection(
    tm_keys(let.providers),
    tm_keys(tm_merge(let.credentials, let.credential_blocks)),
  )
}

content {
  # ... the required_providers and provider blocks above ...

  tm_dynamic "module" {
    condition = tm_length(let.credential_providers) > 0
    labels    = ["providers_credentials"]
    content {
      source              = "${let.modules}/providers-credentials"
      terraform_providers = let.credential_providers
    }
  }
}

A SaaS stack sits outside the account tree, so it declares its own providers the way the region declared AWS, with whatever configuration is not a credential. It still needs an AWS provider, because that is what the lookup module reads Secrets Manager through:

globals "terraform" "providers" "aws" {
  enabled = true
  source  = "hashicorp/aws"
  version = "~> 5.0"
  config = {
    region  = "eu-west-1"
    profile = "acme-shared-services"
  }
}

globals "terraform" "providers" "github" {
  enabled = true
  source  = "integrations/github"
  version = "~> 6.0"
  config = {
    owner = "acme"
  }
}

globals "terraform" "providers" "firehydrant" {
  enabled = true
  source  = "firehydrant/firehydrant"
  version = "~> 0.15"
}

FireHydrant and GitHub have credential entries, so the stack gets the module block and each of those providers gets its references:

// ... required_providers for aws, firehydrant and github; provider "aws" ...

module "providers_credentials" {
  source = "../../../modules/providers-credentials"
  terraform_providers = [
    "firehydrant",
    "github",
  ]
}
provider "firehydrant" {
  api_key = module.providers_credentials.secrets.firehydrant.api_key
}
provider "github" {
  owner = "acme"
  app_auth {
    id              = module.providers_credentials.secrets.github.app_id
    installation_id = module.providers_credentials.secrets.github.app_installation_id
    pem_file        = base64decode(module.providers_credentials.secrets.github.private_key_b64)
  }
}

No stack references a secret identifier, and a stack whose providers need no credentials gets no module block at all.

Reviewing a provider change

Because the output is committed and generation is enforced in continuous integration, a provider change arrives in review as one edit to a declaration plus the exact set of stacks it rewrites. Moving the AWS provider to the next major version is a single version field in stacks/imports/aws/provider.tm.hcl. In the production repository this comes from, that one field feeds several hundred generated provider blocks. The diff that comes with it names every stack that picks the change up, and terramate list --changed on the branch is the list the pipeline will plan.

A major upgrade is still a large mechanical commit, and splitting it by subtree reviews better than landing every stack at once.

Limitations

The single-version rule is Terraform’s, not Terramate’s. A stack that genuinely needs two major versions of one provider cannot be expressed this way, because the alias shares the constraint emitted for the base provider.

Every credential reference is a string until tm_hcl_expression pastes it into the output, so terramate generate will happily render a reference to an output that does not exist.

Attributes come from the declaration, but blocks do not. A provider whose credentials live in a nested block, as GitHub’s app_auth does, needs its own tm_dynamic and condition in the generator. Adding such a provider is a generator edit, not just a global.

What’s next?

Everything in this series so far is for people who write Terraform. The next post turns the same generator machinery towards people who do not: Generating Terraform for teams from one JSON file lets a team describe its Slack channel, GitHub teams and on-call rotation in a team.json, and the monorepo produces the Terraform. It is the point where an infrastructure monorepo starts paying off for developers, not only for the platform team.

Search everything