google-cloud-platformterraformterraform-provider-gcpgoogle-secret-manager

Errors/difficulty with google secret manager and terraform


Edited to add: Very simple answer is below. I was over engingeering.

I'm trying to learn some basic terraform, and have set what i thought was a sound setup.

I separated into two apply's as i was having trouble with some dependencies (and wanting to isolate the issues, which persisted even so). I create service accounts and keys in 1, then in 2 I apply them to Secret Manager/Secret Manager Versions and try to use them.

The issue is that when I run #2 I get errors on resource "google_secret_manager_secret_version" "cr_sec_vers":. They look like those in the table below (Ref #1).

2 Questions:

How should I be passing the secret_data into the secret_manager_version? I'm really struggling.

Is my resource "google_cloud_run_service" "cr_service" seemingly set up right?

Thanks all, sorry for the info overload.

Ref #1

secret_data = Error
data.terraform_remote_state.config1.outputs.cr_sa_key_out.value │ Error: Unsupported attribute ... │ Can't access attributes on a primitive-typed value (string).
jsonencode() ""
base64encode() ""
data.terraform_remote_state.config1.outputs.cr_sa_key_out Error: Error creating SecretVersion: googleapi: got HTTP response code 404 with body:

404. That’s an error.

The requested URL /v1/cr∂-secret-file:addVersion?alt=json was not found on this server. That’s all we know. with google_secret_manager_secret_version.cr_sec_vers, on main.tf line 221, in resource "google_secret_manager_secret_version" "cr_sec_vers": 221: resource "google_secret_manager_secret_version" "cr_sec_vers" {...

base64encode(same) ""
jsonencode(same) ""
jsonencode(data.terraform_remote_state.config1.outputs.cr_sa_key_out_attr) ""
data.terraform_remote_state.config1.outputs.cr_sa_key_out_attr Inappropriate value for attribute "secret_data": string required.

base64encode(same) | Invalid value for "str" parameter: string required. |

Ref #2

Looking in the state file, I see:

"outputs": {
////...////
"cr_sa_key_out": {
      "value": "xxxxxlongkeyxxxxx",
      "type": "string",
      "sensitive": true
    }
}

and

"resources": [
////...////
      "mode": "managed",
      "type": "google_service_account_key",
      "name": "cr_key",
      "provider": "provider[\"registry.terraform.io/hashicorp/google\"]",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "id": "projects/<proj>/serviceAccounts/<sa_name>@<proj>.iam.gserviceaccount.com/keys/<<key_id>>",
            "keepers": null,
            "key_algorithm": "KEY_ALG_RSA_2048",
            "name": "projects/<proj>/serviceAccounts/<sa_name>@<proj>.iam.gserviceaccount.com/keys/<key_id>",
            "private_key": "xxxxPRIVATE_KEYxxxxxx",
            "private_key_type": "TYPE_GOOGLE_CREDENTIALS_FILE",
            "public_key": "xxxxxPUBLIC KEYxxxxx",
            "public_key_data": null,
            "public_key_type": "TYPE_X509_PEM_FILE",
            "service_account_id": "projects/<proj>/serviceAccounts/<sa_name>@<proj>.iam.gserviceaccount.com",
            "valid_after": "<date>",
            "valid_before": "<date>"
          },
          "sensitive_attributes": [],
          "private": "xxxxxx==",
          "dependencies": [
            "google_project_iam_member.sa-accounts-iam",
            "google_service_account.sa",
            "random_id.suffix"
          ]
        }
      ]
    },

Ref #3 (Main.tf 1)

terraform {
  required_version = ">= 1.0"
  backend "gcs" {
    bucket = "z_tf"
  } 
  required_providers {
    google-beta = ">=3.8"
    google = {
      version = "~> 4.0.0"
    }
  }
}

# PROVIDERS

provider "google" {
  project = var.project
  region  = var.region
}

provider "google-beta" {
  project = var.project
  region  = var.region
}

////....////

resource "google_service_account_key" "cr_key" {
  service_account_id = "projects/${var.project}/serviceAccounts/${var.cr_sa}-${random_id.suffix.hex}@${var.project}.iam.gserviceaccount.com"
  depends_on         = [google_project_iam_member.sa-accounts-iam]
  key_algorithm      = "KEY_ALG_RSA_2048"
  private_key_type   = "TYPE_GOOGLE_CREDENTIALS_FILE"
}

output "cr_sa_key_out" {
  value     = google_service_account_key.cr_key.private_key
  sensitive = true
}
output "cr_sa_key_out_attr" {
  value     = google_service_account_key.cr_key
  sensitive = true
}

output "cr_sa_out" {
  value     = google_service_account.sa[var.cr_sa].name
  sensitive = false
}

Ref #4 (Main.tf 2) (second directory)

##<<Same terraform{} and provider{}>>##
# DATA/OUTPUTS

data "terraform_remote_state" "config1" {
  backend = "gcs"
  config = {
    bucket = "z_tf"
  }
}

resource "google_secret_manager_secret" "cr_secret" {
  secret_id = "${var.cr_sa}-key-cr-secret-file"
  replication {
    automatic = true
  }
}

#just something I used to make sure it wasn't a delay.
resource "null_resource" "stall_30_seconds" {
  provisioner "local-exec" {
    command = <<-EOF
      sleep 30
EOF
  }
}

resource "google_secret_manager_secret_version" "cr_sec_vers" {
  secret = google_secret_manager_secret.cr_secret.secret_id
  secret_data = data.terraform_remote_state.config1.outputs.cr_sa_key_out.value
  depends_on  = [null_resource.stall_30_seconds]
}
/////

## Cloud Run
resource "google_cloud_run_service" "cr_service" {
  provider = google-beta
  name     = var.cloud_run_service_name
  location = var.region
  template {
    spec {
      service_account_name = data.terraform_remote_state.config1.outputs.cr_sa_out
      # google_service_account.sa[var.cr_sa].name
      containers {
        image = "ghcr.io/dbt-labs/dbt-bigquery"
        env {
          name = google_secret_manager_secret_version.cr_sec_vers.id
          value_from {
            secret_key_ref {
              name = google_secret_manager_secret_version.cr_sec_vers.name
              key  = "latest" #google_secret_manager_secret_version.cr_sec_vers.name
            }
          }
        }
      }
    }
  }
  traffic {
    percent         = 100
    latest_revision = true
  }
}

Solution

  • Don't put the secret version in secret manager. Create your secret, use that dependency to configure other resources (...), but fill the secret value (the version) manually.

    The reasons are: