terraform

terraform retrieve file from URL

Here my concrete problem but the question is more generic: I'm creating a VM through a Terraform file with a certain cloud-init template.

resource "openstack_compute_instance_v2" "VM" {
....
 
  user_data = file('cloud-init')

}

Now , I'd like to re-use a cloud-init file that is external to the repository, let's say at https://site/cloud-init

  user_data = fromUrl('https://server/cloud-init')  #something like this

Is there a generic simple way to retrieve a file from an URL and uses it's contents? Googling got me nowhere and I feel like the problem is generic enough.

I'd want to avoid having to 'curl it' myself.

Update: I don't necessarily mean an HTTP URL, could be a git, ssh ... URL.

Solution

  • You can use two approaches based on the type of content:

    1. Use the http data source
    2. Use terraform_data resource

    For the http data source, two things are important:

    At present this resource can only retrieve data from URLs that respond with text/* or application/json content types, and expects the result to be UTF-8 encoded regardless of the returned content type header.

    The second one is related to the security:

    Although https URLs can be used, there is currently no mechanism to authenticate the remote server except for general verification of the server certificate's chain of trust. Data retrieved from servers not under your control should be treated as untrustworthy.

    If that is acceptable, then you can use the http data source, something like:

    data "http" "cloud_init" {
  url = "https://server/cloud-init"

  # Optional request headers
  request_headers = {
    Accept = "text/yaml"
  }
}

    Then, since this will be a YML file, you would have to decode it using yamldecode built-in function:

    resource "openstack_compute_instance_v2" "VM" {
....
 
  user_data = yamldecode(data.http.cloud_init.response_body)

}

    The second approach is basically allowing you to run a shell command/script (which you would otherwise run manually) and download the file:

    resource "terraform_data" "cloud_init" {
  triggers_replace = [] # you could tie downloading a file to some kind of a change in other resources 

  provisioner "local-exec" {
    command = "curl -s -o ${path.root}/cloud-init.yaml https://server/cloud-init"
  }
}

resource "openstack_compute_instance_v2" "VM" {
....
 
  user_data = file("${path.root}/cloud-init.yaml")


  depends_on = [terraform_data.cloud_init]
}

    As a side note, the second approach creates an explicit dependency between resources with depends_on, so the first approach is probably a bit more elegant.