terraformhclconfluent-cloud

Terraform YAML config - object does not have an attribute names


I'm pulling my hair out trying to debug an issue where I have a yaml file which contains two attributes per entry:

topics:
  - name: topic_one
    partitions: 24
  - name: topic_two
    partitions: 1
  - name: topic_three
    partitions: 12
  - name: topic_four
    partitions: 6

I'm pulling this into TF with:

locals {
  confluent = yamldecode(file("./confluent.yaml"))["topics"]

  confluent_topics = flatten([for topic in local.confluent :
  {
    "name"       = topic.name
    "partitions" = topic.partitions
  }])
}

Passing into the module with:

module "confluent" {
  source = "./modules/confluent-cloud"
  count  = var.confluent_cluster ? 1 : 0
  topics = local.confluent_topics
  
  region = var.region
}

Then in the module:

resource "confluent_kafka_topic" "kafka" {
  for_each = {
    for topic in var.topics : topic.name => name
  }  

  kafka_cluster {
    id = confluent_kafka_cluster.kafka.id
  }
  topic_name = each.value.name
  partitions_count = each.value.partitions
  ...

I've tried just about every combination, and I've set the variable type in the modules variables.tf to just about everything imaginable.. but the error is that it doesn't think the object has an attribute named partitions - I've tried simplifying the local without flattening the list, I've tried so many different variations.. but always something like this;

│ Error: Unsupported attribute
│
│   on main.tf line 40, in locals:
│   40:     "partitions" = topic.partitions
│
│ This object does not have an attribute named "partitions".

Console gives me:

> local.confluent
[
  {
    "name" = "topic_one"
    "partitions" = 24
  },
  {
    "name" = "topic_two"
    "partitions" = 1
  },
  {
    "name" = "topic_three"
    "partitions" = 12
  },
  {
    "name" = "topic_four"
    "partitions" = 6
  }

Console on the flattened local just returns:

> local.confluent_topics
(known after apply)

...which is somewhat puzzling because I think it should know this without apply, however.. I'm fairly confident this is not my issue.

The annoying thing is I've done this before without issue.. so I'm wondering whether this is a TF Version situation where something has to be handled differently?

❯ terraform version
Terraform v1.1.7
on darwin_arm64
+ provider registry.terraform.io/confluentinc/confluent v0.7.0
+ provider registry.terraform.io/hashicorp/aws v4.12.1
+ provider registry.terraform.io/hashicorp/kubernetes v2.11.0
+ provider registry.terraform.io/hashicorp/null v3.1.1
+ provider registry.terraform.io/hashicorp/random v3.1.3
+ provider registry.terraform.io/hashicorp/tls v3.3.0

Your version of Terraform is out of date! The latest version
is 1.1.9. You can update by downloading from https://www.terraform.io/downloads.html

Solution

  • Your logic is all well, except the looping with attribute name. You can't have the name attribute & fetch the partitions field from it. It can be only fetched from topic object.

    Try this..

    resource "confluent_kafka_topic" "kafka" {
      for_each = {
        for topic in var.topics : topic.name => topic
      }  
    
      kafka_cluster {
        id = confluent_kafka_cluster.kafka.id
      }
      topic_name = each.value.name
      partitions_count = each.value.partitions
      ...
    

    You have the unique key in the form of topic.name & topic is the object which has partitions field.