Create a nested (2 level) map in Terraform output

I have a Terraform module that create EC2 instances "for me" grouped by "cluster" with the following output (here in a cluster_module_output local for easy debug/testing):

locals {
  cluster_module_output = {
    # API cluster on eu-west-1a:
    some_unique_index_1 = {
      cluster_az_name = "eu-west-1a"
      cluster_name = "api"
      instance_names = [
        "api-1a-node_1",
        "api-1a-node_2",
      ]
      instances_ids = […]
    }
    # Web cluster on eu-west-1a:
    some_unique_index_2 = {
      cluster_az_name = "eu-west-1a"
      cluster_name = "web"
      instance_names = [
        "web-1a-node_1",
        "web-1a-node_2",
        "web-1a-node_3",
      ]
      instances_ids = […]
    }
    # API cluster on eu-west-1b:
    some_unique_index_3 = {
      cluster_az_name = "eu-west-1b"
      cluster_name = "api"
      instance_names = [
        "api-1b-node_1",
        "api-1b-node_2",
      ]
      instances_ids = […]
    }
  }
}

In my Terraform root module I would like to output various informations, such as:

• All instances names (no matter the cluster)
• All instances names, grouped by cluster and by AZ

For the former (raw list of instances), I can do:

output "instances_names" {
  value = flatten([for cluster in local.cluster_module_output : cluster.instance_names])
}

Which gives:

instances_names = [
  "api-1a-node_1",
  "api-1a-node_2",
  "api-1b-node_1",
  "api-1b-node_2",
  "web-1a-node_1",
  "web-1a-node_2",
  "web-1a-node_3",
]

But for the latter (instances grouped by cluster and by AZ) I fail to find an expression.

Desired output:

instances_names_by_cluster = {
  "api" = {
    "eu-west-1a" = [
      "api-1a-node_1",
      "api-1a-node_2",
    ]
    "eu-west-1b" = [
      "api-1b-node_1",
      "api-1b-node_2",
    ]
  }
  "web" = {
    "eu-west-1a" = [
      "web-1a-node_1",
      "web-1a-node_2",
      "web-1a-node_3",
    ]
  }
}

My failing attempts are:

output "attempt1" {
  value = tomap({
    for item in (
      flatten([
        for cluster in local.cluster_module_output : {
          cluster_name = cluster.cluster_name
          cluster_az = cluster.cluster_az_name
          instance_ids = cluster.instances_ids
        }
      ])
    ) : item.cluster_name => {
      (item.cluster_az) = item.instance_ids
    }
  })
}

Fails with:

Error: Duplicate object key:

Two different items produced the key "api" in this 'for' expression. If duplicates are expected, use the ellipsis (...) after the value expression to enable grouping by key

Obviously I need a way to merge first-level elements in the loop.

Adding the ellipsis (...) like advised:

output "attempt2" {
  value = tomap({
    for item in (
      flatten([
        for cluster in local.cluster_module_output : {
          cluster_name = cluster.cluster_name
          cluster_az = cluster.cluster_az_name
          instance_ids = cluster.instances_ids
        }
      ])
    ) : item.cluster_name => {
      (item.cluster_az) = item.instance_ids
    }... # ellipsis added here
  })
}

Raises no more error but don't have the desired output:

attempt2 = {
  "api" = [
    {
      eu-west-1a = [
        "api-1a-node_1",
        "api-1a-node_2",
      ]
    },
    {
      eu-west-1b = [
        "api-1b-node_1",
        "api-1b-node_2",
      ]
    },
  ]
  "web" = [
    {
      eu-west-1a = [
        "web-1a-node_1",
        "web-1a-node_2",
        "web-1a-node_3",
      ]
    },
  ]
}

How can I do this? Do I have to use intermediate locals?

Your last attempt is very close: you only need to merge a list of objects into a single object. merge does exactly that. You have a list of values, merge expects separate arguments, so splat comes to rescue.

However, you can also get rid of that convoluted flatten:

desired_output = {
    for k, v in {
        for item in values(local.cluster_module_output):
        # Group by cluster_name
        item.cluster_name => {
            (item.cluster_az_name): item.instance_names
        }...
    }: k => merge(v...)
}

Note: this expects that (cluster_name, cluster_az_name) pairs are unique in your input - there should be no values with both components equal.