terraform

How to validate configuration in Terraform before deployment?

I have a configuration of a service called loki.yml. I do not want to deploy invalid configuration. To check the configuration, I should execute the command loki -log.level debug -verify-config -config.file ./loki.yml.

How should I write Terraform configuration to execute this command?

I tried reading:

and researching:

Could someone share how this should be written? Are there best practices for configuration validation - should this maybe be preferred to be done outside terraform with a Makefile? This is my current configuration:

check "loki_config_is_ok" {
  data "??" "??" {
    ?? get exit code of command ??
  }

  assert {
    condition = ??.exit_code is not 0
    error_message = "dont"
  }
}

resource "nomad_job" "loki" {
  jobspec = file("./loki.nomad.hcl")
  hcl2 {
    vars     = { mark = file("./loki.yml") }
  }
  # disallow deployment if config is invalid ?
  depends_on = [loki_config_is_ok]
}
Solution

  • Only validation

    You can prevent deploying an invalid configuration using a plain data "external" block if you're OK with just failing the plan step entirely when validation fails. Assuming loki indicates success by exit code (which it should and seems to), the following will be sufficient:

    data "external" "loki_config_validation" {
    program = ["loki", "-verify-config", ...]  # whatever args you need
}

    Yes, that's it. I'm not ready to install loki toolset now, so let's use simple exit 1 script instead:

    data "external" "loki_config_validation" {
    program = ["bash", "-c", "echo FAIL >&2 && exit 1"]
}

    Using terraform 1.9.4 and provider hashicorp/external 2.3.3 I get the following output block when running terraform plan:

    
╷
│ Error: External Program Execution Failed
│ 
│   with data.external.loki_config,
│   on data.tf line 26, in data "external" "loki_config":
│   26:   program = ["bash", "-c", "echo FAIL >&2 && exit 1"]
│ 
│ The data source received an unexpected error while attempting to execute the program.
│ 
│ Program: /usr/bin/bash
│ Error Message: FAIL
│ 
│ State: exit status 1
╵

    Plan is not generated, this is a hard failure. So if your loki command exits with non-zero code, validation fails and deployment is prevented. You don't need any shenanigans to validate this configuration. You don't need related assert or precondition blocks either - it will just abort planning entirely on failure.

    It will also display the stderr of that command on failure. "Error Message:" line will contain the error. If your program writes to stdout, you can simply redirect it with >&2.

    Capturing output

    First, I agree that a script in this answer is a good solution to the general problem "I want to run an external script with arguments and read the stdout". I'm not even sure it's worth introducing a provider dependency.

    However, there are existing (and quite popular, according to their download stats) providers as well. I found these two: data and resource.

    They are not a silver bullet. These modules support capturing exit code and stdout/stderr, also providing you a choice whether to abort on failure. However, shell escaping now comes into play: when your arguments aren't "simple" (e.g. there's no need to escape a resource ID), you need a generic escaper. If you want to run in a fully managed manner, here's a null resource from the same provider that performs shell escaping. It also handles \ vs \\ problem (note that it isn't the provider messing up with slashes - it's the shell interpreting \\ as a single backslash).

    Note that this module runs in a shell, as opposed to data "external" invoking an executable directly.

    It would look something like this:

    module "shell_data_hello" {
  source  = "Invicton-Labs/shell-data/external"
  command_unix = <<EOF
  echo 'failed \\ slash' ${module.escape_var.unix} >&2 && exit 1
EOF
}

module "escape_var" {
  source = "Invicton-Labs/shell-escape/null"
  string = var.to_be_escaped
}

variable "to_be_escaped" {
  type = string
  default = "Hi! See $ and \\ characters and _ _ spaces"
}

output "command_out" {
  value = module.shell_data_hello.stdout
}

    Running plan on this produces:

    ╷
│ Error: External Program Execution Failed
│ 
│   with module.shell_data_hello.data.external.run,
│   on .terraform/modules/shell_data_hello/main.tf line 76, in data "external" "run":
│   76:   program = local.is_windows ? ["powershell.exe", "${abspath(path.module)}/run.ps1"] : [local.var_unix_interpreter, "${abspath(path.module)}/run.sh"]
│ 
│ The data source received an unexpected error while attempting to execute the program.
│ 
│ Program: /bin/sh
│ Error Message: failed \ slash Hi! See $ and \ characters and _ _ spaces
│ State: exit status 1

    Replacing exit 1 with exit 0 and removing stderr redirection produces expected output:

    Changes to Outputs:
  + command_out    = "failed \\ slash Hi! See $ and \\ characters and _ _ spaces"

    Make sure not to over-escape: if there are some shell syntax elements, you shouldn't escape them.