I am trying the dynamically manipulate the tfvars file using golang.
Here is my code
package main
import (
"fmt"
"io"
"log"
"os"
"github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/gohcl"
"github.com/hashicorp/hcl/v2/hclsyntax"
"github.com/hashicorp/hcl/v2/hclwrite"
)
type RawDatabase struct {
Name string `hcl:"name"`
BusinessVerticalID string `hcl:"business_vertical_id"`
Readers []string `hcl:"readers"`
Contributors []string `hcl:"contributors"`
}
type Config struct {
Environment string `hcl:"environment"`
BusinessEntity *string `hcl:"business_entity"`
EntitlementLookup map[string]string `hcl:"entitlement_lookup"`
RawDatabases []RawDatabase `hcl:"raw_databases"`
}
func readFile(filePath string) ([]byte, error) {
// Open the file for reading
file, err := os.Open(filePath)
if err != nil {
return nil, err
}
defer file.Close()
// Read the file content into a byte slice
content, err := io.ReadAll(file)
if err != nil {
return nil, err
}
return content, nil
}
func writeFile(data []byte, filePath string) ( error) {
// Open the file for reading
file, err := os.OpenFile(filePath, os.O_WRONLY|os.O_CREATE, 0644)
if err != nil {
return err
}
defer file.Close()
// Read the file content into a byte slice
_, err = file.Write(data)
if err != nil {
return err
}
return nil
}
func main() {
// Read the tfvar file content
content, err := readFile("demo.tfvars")
if err != nil {
log.Fatalf("Error reading tfvars file: %v", err)
}
// Parse tfvars content
parsedObject, diags := hclsyntax.ParseConfig(content, "", hcl.Pos{})
if diags.HasErrors() {
log.Fatalf("Failed to parse tfvars content: %s", diags)
}
// Print parsedObject.Body to debug
fmt.Println("Parsed Body:", parsedObject.Body)
// Decode HCL into Go struct
var config Config
diags = gohcl.DecodeBody(parsedObject.Body, nil, &config)
if diags.HasErrors() {
log.Fatalf("Failed to decode tfvars content: %s", diags)
}
// Print Go struct
fmt.Printf("%+v\n", config)
// Convert Go struct back to HCL
f := hclwrite.NewEmptyFile()
gohcl.EncodeIntoBody(&config, f.Body())
// Write HCL content to a file
err = writeFile(f.Bytes(), "../../../terraform/project1/dev_out.tfvars")
if err != nil {
log.Fatalf("Failed to write HCL to file: %v", err)
}
}
and the demo.tfvars file content is
environment = "dev"
entitlement_lookup = {
"obj1" = "obj_id1",
"obj2" = "obj_id2",
}
raw_databases = [
{
name = "db1"
business_vertical_id = "fin"
readers = []
contributors = ["obj1", "obj2"]
},
{
name = "db2"
business_vertical_id = "fin"
readers = []
contributors = ["obj2"]
},
{
name = "db3"
business_vertical_id = "fin"
readers = []
contributors = ["obj1"]
}
]
My code is working fine if i try to the above code without raw_database.
Help me to solve the error
panic: unsuitable DecodeExpression target: no cty.Type for main.RawDatabase (no cty field tags)
goroutine 1 [running]:
github.com/hashicorp/hcl/v2/gohcl.DecodeExpression({0x7fdab078cc68, 0xc0000fa8c0}, 0x5f54e0?, {0xc0000bbc80, 0xc0000bb9a0})
/go/pkg/mod/github.com/hashicorp/hcl/v2@v2.20.1/gohcl/decode.go:296 +0x7f4
github.com/hashicorp/hcl/v2/gohcl.decodeBodyToStruct({0x68cb58, 0xc000136420}, 0x0, {0x615b80?, 0xc0000bb980?, 0xc00019fe00?})
/go/pkg/mod/github.com/hashicorp/hcl/v2@v2.20.1/gohcl/decode.go:127 +0xd38
github.com/hashicorp/hcl/v2/gohcl.decodeBodyToValue({0x68cb58, 0xc000136420}, 0x0, {0x615b80?, 0xc0000bb980?, 0xc00019fe78?})
/go/pkg/mod/github.com/hashicorp/hcl/v2@v2.20.1/gohcl/decode.go:46 +0xba
github.com/hashicorp/hcl/v2/gohcl.DecodeBody({0x68cb58, 0xc000136420}, 0x0, {0x5f2060?, 0xc0000bb980?})
/go/pkg/mod/github.com/hashicorp/hcl/v2@v2.20.1/gohcl/decode.go:39 +0xc5
main.main()
/workspaces/cdassp/go/cmd/go_hcl/main.go:106 +0x1bd
exit status 2
I have verified below
Double-check tag placement: Ensure each hcl:"..." tag is placed directly above the corresponding struct field. Incorrect placement can lead to the error.
Verify tag syntax: Make sure the tag format is correct: hcl:"key_name". The key name should match the exact name used in your HCL configuration file for that field.
Does this mean the struct objects can not be parsed with HCL?
When using the gohcl abstraction to declare HCL schema using Go struct tags, the HCL tags can only represent concepts that can map to HCL's "body schema" model. That means:
HCL itself does not model attribute types or values. Instead, it delegates that to an upstream library called cty. You can see that in HCL's low-level API in that hcl.Expression's Value method returns cty.Value, not an HCL-specific type. (Disclosure: I am the primary author and maintainer of cty.)
To support the gohcl abstraction, HCL delegates attribute value decoding to cty's corresponding package gocty, which has its own rules for mapping cty.Value to "normal" Go types, including its own struct tags for describing object types.
That means that if you want to decode an object-typed value into an instance of a Go struct type then you'll need to declare that struct type with gocty's struct tags, rather than gohcl's struct tags. gohcl's struct tags are only for decoding HCL's own concepts: attributes and nested blocks.
The following pair of types should achieve the effect you wanted:
type RawDatabase struct {
Name string `cty:"name"`
BusinessVerticalID string `cty:"business_vertical_id"`
Readers []string `cty:"readers"`
Contributors []string `cty:"contributors"`
}
type Config struct {
Environment string `hcl:"environment"`
BusinessEntity *string `hcl:"business_entity"`
EntitlementLookup map[string]string `hcl:"entitlement_lookup"`
RawDatabases []RawDatabase `hcl:"raw_databases"`
}
Notice that RawDatabase now has cty: struct tags, instead of hcl: struct tags. The tags in Config tell gohcl which attributes to request in the hcl.BodySchema object it generates. The type of RawDatabases, and the field tags in RawDatabase, tell gocty to expect a list of objects that each have the four attributes you specified.
Terraform itself does not use the gohcl abstraction to implement its .tfvars format. Instead, it works directly with the low-level HCL API.
Terraform's interpretation of such a file works roughly like this:
file, diags := hclsyntax.ParseConfig([]byte(src), "filename.tfvars", hcl.InitialPos)
if diags.HasErrors() {
// (handle the errors)
}
attrs, diags := file.Body.JustAttributes()
if diags.HasErrors() {
// (handle the errors)
}
vals := make(map[string]cty.Value, len(attrs))
for name, attr := range attrs {
vals[name], diags = attr.Expr.Value(nil)
if diags.HasErrors() {
// (handle the errors)
}
}
The result in vals is a map with an element for each of the defined variables, where each one is represented as cty.Value.
Terraform never needs to translate those values into Go string, slice, or struct types because it does all of its work using the cty.Value API, because that represents the Terraform language type system.
However, if you need to do that for your own purposes then you can still use gocty with the values if you like, passing each value into gocty.FromCtyValue with an appropriate target type.
However, if your goals allow for your interpretation of the file to be stricter than Terraform would be then your original approach of using gohcl along with gocty, with the modifications I proposed above, are a reasonable shortcut. I mention this other detail just to avoid presenting the misleading impression that Terraform's .tfvars format is decoded using the gohcl helper. (gohcl is intended for applications with simpler needs than Terraform.)