How to call text files as user-data in Terraform without UTF-8 encoding?

Terraform version: v1.0.4

I am experimenting with user_data attribute for EC2 instances. Here are the contents of a .txt file that I am trying to use at the moment,

#cloud-config
aws_image: true

Following are the scenarios that I have tested

1. Using 'locals'. Works fine.

2. Converting file to UTF-8 encoding and then making use of templates, Works fine.

3. Using a .txt file without UTF-8 encoding, this is the problematic scenario, user-data section for defined EC2 instance turns up empty. I have tested few permutations,

3.1.

data "template_file" "user_data_test"{
  template = filebase64("/Users/home/Desktop/centos_ec2_aws/test-user-data.txt")
}

resource "aws_instance" "my_aws_Linux_vm" {
  .
  .
  user_data_base64 = data.template_file.user_data_test.template
}

3.2.

data "template_file" "user_data_test{
  template = filebase64("/Users/home/Desktop/centos_ec2_aws/test-user-data.txt")
}
resource "aws_instance" "my_aws_Linux_vm" {
  .
  .
  user_data = data.template_file.user_data_test.template
}

I am looking for an approach where even if I don't use a .txt or .sh file with UTF-8, my deployment completes without any error. Let me know if this is doable.

Because template files are written in the Terraform language's template syntax, the template itself must always be UTF-8 encoded in order to be valid syntax, but you can in principle write a template that contains expressions which will produce base64 data and then assign that result to user_data_base64. With that said, I'm having trouble imagining what a use-case would be for that and you didn't share one so I can't give a real example.

If your intended result is still text but using some other character encoding than UTF-8 then an alternative approach would be to have the template and the template result both be UTF-8 but then to convert the resulting string to some other character encoding afterwards.

Here's an example where I've also replaced your use of the deprecated data "template_file" with the modern templatefile function:

locals {
  user_data = templatefile("${path.module}/test-user-data.txt", {})
}

resource "aws_instance" "my_aws_Linux_vm" {
  # ...

  user_data_base64 = textencodebase64(local.user_data, "UTF-16LE")
}

The above uses textencodebase64 to translate from UTF-8 to UTF-16LE encoding.

Alternatively, if your goal is just to send a file verbatim to EC2 without Terraform doing any template processing on it at all, you can use filebase64 directly in the user_data_base64 argument:

  user_data_base64 = filebase64("${path.module}/test-user-data.txt")