I would like to check if I need to run an installation script stored in the s3 bucket. If the configuration does have an s3 bucket, then download and run the installer
in my variables.tf, I have
variable "util_apps" {
type = map(any)
default = {
"install_list" = [
{
"s3_bucket" = "crowdstrike"
"setup_command" = "install.sh"
},
{
"setup_command" = "pip3 install boto3"
}
]
}
}
in my main.tf, I have:
user_data = templatefile("init.tftpl", var.util_apps)
and finally in init.tftpl I have a piece of code like
%{for itm in install_list }
key_exists = %{ if contains(keys(itm), "s3_bucket") } "yes" %{else} "no" %{endif}
if [ $key_exists == "yes" ]
aws s3 cp s3://${s3_bucket}/${setup_command} ./
chmod +x ${setup_command}
./${setup_command}
fi
# other stuffs ...
%{endfor}
I end up with an error like:
init.tftpl:21,41-51: Unsupported attribute; This object does not have an attribute named "s3_bucket".
I have tried various ways of checking of the key exists in the map, but they dont work in the context of user-data, apparently. I know I can define a special value for "s3_bucket", like "none", but I want to exhaust other options first
I think your template here has some confusion about what parts are being evaluated in Terraform as part of the template vs. which parts are being evaluated at runtime by the shell that's running this script.
Because both Terraform templates and Unix shells overlapping syntax ${ ... } to represent interpolation, generating a shell script requires some extra care both to make sure to escape interpolation sequences that should be evaluated by Bash and to remember as you are writing which data is visible to Terraform and which data is visible to Bash.
In the example you shared:
The %{ if contains ... } construct is being evaluated by Terraform, and so itm in that expression refers to a value in the current template scope.
That overall line is defining a shell variable called key_exists which is only visible to the shell and not to Terraform.
However, the aws s3 cp command includes some Terraform template interpolation sequences that refer to names that don't seem to be in scope for Terraform: s3_bucket and setup_command.
If you want to refer to these in the template context then you'd need to specify itm.s3_bucket and itm.setup_command instead, but note that this part of the template isn't guarded by your if contains... condition and so Terraform will try to evaluate it unconditionally. You'll need to restructure the template to ensure that you try to interpolate those values only when they are present.
If you want to refer to these in the shell context then you'd need to escape the interpolation sequences so that Terraform will ignore them and the shell can evaluate them instead: s3://$${s3_bucket}/$${setup_command}, where Terraform will replace $${ with just ${ so that it will be valid syntax for the shell. This will work only if you've already assigned values to these earlier in the generated shell script.
It gets pretty confusing to generate source code in one language using code written in another language via template interpolation like this, and so I would suggest a different approach: generate only the part of the script that is assigning values to shell variables, and then make the actual script body be totally literal.
Here is an example of generating Unix-shell-style syntax from a map of strings in the Terraform language:
locals {
default_var_values = {
s3_bucket = ""
setup_command = ""
}
install_vars_shell = [
for vars in var.util_apps.install_list : <<-EOT
%{ for k, default_v in local.default_var_values }
%{k}='${replace(try(vars[k], default_v), "'", "'\\''")}'
%{ endfor }
EOT
]
}
With the default values you showed for var.util_apps in your question, local.install_vars_shell will be something like the following:
[
<<-EOT
s3_bucket='crowdstrike'
setup_command='install.sh'
EOT
,
<<-EOT
s3_bucket=''
setup_command='pip3 install boto3'
EOT
]
This should be valid shell syntax for just the problem of making your Terraform map elements be visible as variables in the shell scope at runtime. That means you can write the rest of the script as an entirely static file referring to those variables, and thus not need to worry about any conflicts between Terraform's template syntax and the shell syntax:
user_data = <<EOT
%{ for vars_shell in local.util_apps ~}
${vars_shell}
${file("${path.module}/init.sh")}
############################################
%{ endfor }
EOT
If your init.sh file were, for the sake of example, just the following:
echo "Setup command is ${setup_command}"
...then the final value user_data would be something equivalent to:
s3_bucket='crowdstrike'
setup_command='install.sh'
echo "Setup command is ${setup_command}"
############################################
s3_bucket=''
setup_command='pip3 install boto3'
echo "Setup command is ${setup_command}"
############################################
I just used a very simple script here because my focus is on showing the Terraform part of this technique, but you can use any shell syntax you like inside init.sh, including conditional statements and loops, without any special escaping because Terraform will not interpret the contents of init.sh at all.
A final sidebar: you declared your variable here as map(any), where any means that Terraform should analyze the value passed and automatically substitute a specific type to replace any.
With the default value you showed here, I think Terraform will infer the following type constraint:
type = map(list(map(string)))
I expect that the above doesn't match what you intended that value's type to be understood as. In particular, if you add any new keys alongside install_list in future then they would be forced to also be lists of maps of strings, because all elements of a map must have the same type.
To avoid confusion and design problems later on, I'd strongly recommend writing out exactly the type you mean here and not relying on the any placeholder at all. That way Terraform will not need to guess what you mean and can give you better feedback if something goes wrong as you evolve your module in future.
Based on how you're using this variable, I think the following would be the type that most accurately matches your intention:
type = object({
install_list = list(map(string))
})
This declares that your module effectively requires the install_list attribute to be present (the module will be invalid if not, because it refers directly to it) and that it's an object attribute with a specified type.
If you wanted to add another attribute alongside install_list in future then you could do so with any type you like, because each attribute of an object type can have its own distinct type, whereas all elements of a map must have the same type.
This type constraint part is not crucial to what I recommended in the earlier parts above but I'm suggesting it because with a complicated data structure like this I expect you'll probably run into some errors along the way adapting what I suggested into a fully-working solution, and being specific about what type you are expecting will help Terraform give you better feedback when something goes wrong.