I have a Terraform variables.tf in the module root with contents similar to this:
variable "parameters" {
description = "Parameter Store key/values"
type = map(string)
default = {
"/customer1/prod/keycloak/password" = "password123"
"/customer1/prod/keycloak/realm" = "default"
"/customer1/prod/keycloak/url" = "https://customer1-sso.app.something.cloud/auth"
"/customer1/prod/keycloak/userid" = "f:48c1ce1f:monitoring"
"/customer1/uat/keycloak/password" = "password123"
"/customer1/uat/keycloak/realm" = "uat"
"/customer1/uat/keycloak/url" = "https://customer1-sso.npr.app.something.cloud/auth"
"/customer1/uat/keycloak/userid" = "f:d48d4452:monitoring"
"/customer1/uat2/keycloak/password" = "password123"
"/customer1/uat2/keycloak/realm" = "uat2"
"/customer1/uat2/keycloak/url" = "https://customer1-sso.npr.app.something.cloud/auth"
"/customer1/uat2/keycloak/userid" = "f:5fe762fd:monitoring"
"/customer2/prod/keycloak/password" = "password123"
"/customer2/prod/keycloak/realm" = "default"
"/customer2/prod/keycloak/url" = "https://customer2-sso.app.something.cloud/auth"
"/customer2/prod/keycloak/userid" = "f:a053a488:monitoring"
"/customer2/uat/keycloak/password" = "password123"
"/customer2/uat/keycloak/realm" = "uat"
"/customer2/uat/keycloak/url" = "https://customer2-sso.npr.app.something.cloud/auth"
"/customer2/uat/keycloak/userid" = "f:225118c6:monitoring"
"/customer2/uat2/keycloak/password" = "password123"
"/customer2/uat2/keycloak/realm" = "uat2"
"/customer2/uat2/keycloak/url" = "https://customer2-sso.npr.app.something.cloud/auth"
"/customer2/uat2/keycloak/userid" = "f:9d8b2d30:monitoring
}
}
I would like to convert it into a map of objects that looks like this:
{
"customer1_prod" = {
customer = "customer1"
password = "password123"
realm = "default"
url = "https://customer1-sso.app.something.cloud/auth"
userid = "f:48c1ce1f:monitoring"
}
...
}
I have tried this in the root module (main.tf) code:
# Transform flat map of strings into a grouped map of customer-env -> { password, realm, url, userid }
locals {
# Step 1: Flatten parameters to tuples of [group_key, key_in_object, value]
parameter_tuples = [
for full_key, value in var.parameters : {
group_key = join("_", slice(split("/", full_key), 1, 3)) # e.g. customer1_prod
key = split("/", full_key)[3] # e.g. password
value = value
}
]
# Step 2: Group by environment and assemble objects with customer field
customers = {
for group_key in distinct([for p in local.parameter_tuples : p.group_key]) :
group_key => merge(
{
customer = split("_", group_key)[0]
},
merge([
for p in local.parameter_tuples : {
for inner in [p] :
inner.key => inner.value
} if p.group_key == group_key
]...)
)
}
}
output "all_customers" {
value = local.customers
}
module "dd_synthetic_browser" {
source = "./modules/dd_synthetic_browser"
customers = local.customers
}
However, the output shows this:
+ all_customers = {
+ customer1_prod = {
+ customer = "customer1"
+ keycloak = "f:48c1ce1f:monitoring"
}
...
...
How to fix this incorrect map of objects? There are two errors:
keycloak = "f:48c1ce1f:monitoring"
item should be userid = "f:48c1ce1f:monitoring"
Also, I am open to suggestions of better Terraform code to perform this conversion because I think my code is too complicated for its task and could probably be accomplished in a simpler fashion with less lines of code.
Let's reduce the volume of inputs to the minimum to investigate:
variable "parameters" {
description = "Parameter Store key/values"
type = map(string)
default = {
"/customer1/prod/keycloak/password" = "password123"
"/customer1/prod/keycloak/realm" = "default"
"/customer1/prod/keycloak/url" = "https://customer1-sso.app.something.cloud/auth"
"/customer1/prod/keycloak/userid" = "f:48c1ce1f:monitoring"
}
}
And take a look at the intermediary variable parameter_tuples
:
locals {
parameter_tuples = [
for full_key, value in var.parameters : {
group_key = join("_", slice(split("/", full_key), 1, 3)) # e.g. customer1_prod
key = split("/", full_key)[3] # e.g. password
value = value
}
]
}
output "parameter_tuples" {
value = local.parameter_tuples
}
We get the following:
$ terraform plan
Changes to Outputs:
+ parameter_tuples = [
+ {
+ group_key = "customer1_prod"
+ key = "keycloak"
+ value = "password123"
},
+ {
+ group_key = "customer1_prod"
+ key = "keycloak"
+ value = "default"
},
+ {
+ group_key = "customer1_prod"
+ key = "keycloak"
+ value = "https://customer1-sso.app.something.cloud/auth"
},
+ {
+ group_key = "customer1_prod"
+ key = "keycloak"
+ value = "f:48c1ce1f:monitoring"
},
]
The key is 'keycloak' for all tuples because split("/", full_key)
, passed a string like /customer1/prod/keycloak/realm
, will return data of the form:
[
"",
"customer1",
"prod",
"keycloak", # <- 3
"realm",
]
So just bumping that index:
locals {
parameter_tuples = [
for full_key, value in var.parameters : {
group_key = join("_", slice(split("/", full_key), 1, 3)) # e.g. customer1_prod
key = split("/", full_key)[4] # 3->4
value = value
}
]
customers = {
for group_key in distinct([for p in local.parameter_tuples : p.group_key]) :
group_key => merge(
{
customer = split("_", group_key)[0]
},
merge([
for p in local.parameter_tuples : {
for inner in [p] :
inner.key => inner.value
} if p.group_key == group_key
]...)
)
}
}
output "all_customers" {
value = local.customers
}
produces the desired output:
$ terraform plan
Changes to Outputs:
+ all_customers = {
+ customer1_prod = {
+ customer = "customer1"
+ password = "password123"
+ realm = "default"
+ url = "https://customer1-sso.app.something.cloud/auth"
+ userid = "f:48c1ce1f:monitoring"
}
}