Terraform resource creation from yaml input file

Looking for a way to automate the creation of Collection and index creation in Azure Cosmos DB in a dynamic way, where our users can give the inputs in yaml format with the collection properties, and this should be capable to create the collection and index as they required.

The inputs file should have all the track of these properties and Terraform code should be align with the new collection/index items when adding (need to have the terraform state file in sync with what users adding)

below is the terraform code created for this purpose and I have a working [yaml parser][1] script in my azuredevops pipeline, which will convert the given inputs in the yaml files to variables.

But here I am not getting a way to get succeeded to update above with the terraform code

resource "azurerm_cosmosdb_mongo_collection" "collection" {  
  for_each = { for coll in var.collections : coll.name => coll }  
  
  name                = each.value.name  
  resource_group_name = var.cosmosdb_account_rg  
  account_name        = var.cosmosdb_account_name  
  database_name       = each.value.database_name  
  
  default_ttl_seconds    = each.value.default_ttl_seconds  
  shard_key              = each.value.shard_key  
  analytical_storage_ttl = var.analytical_storage_ttl  
  
  dynamic "index" {  
    for_each = each.value.indexes  
    content {  
      keys   = index.value  
      unique = false  
    }  
  }  
  
  throughput = var.autoscaling_enabled ? null : var.max_throughput  
  
  dynamic "autoscale_settings" {  
    for_each = var.autoscaling_enabled ? [1] : []  
    content {  
      max_throughput = var.max_throughput  
    }  
  }  
}  

Expected input in the yaml file

-  database_name  : "mydb1"
   create_db: true
   - collectionName : "mycollection1"
     create_collection : true
     shard_key      : "_id"
     default_ttl_seconds:  -1
     additional_indexes :
         - keys: "index1"
           unique:  false
         - keys:  "Index2",
           unique:  false
   
   - collectionName : "mycollection1"
     create_collection : true
     shard_key      : "_id"
     default_ttl_seconds:  -1
     additional_indexes :
         - keys: "index1"
           unique:  false
         - keys:  "Index2",
           unique:  false

-  database_name  : "mydb2"
   create_db: true
   - collectionName : "mycollection1"
     create_collection : true
     shard_key      : "_id"
     default_ttl_seconds:  -1
     additional_indexes :
         - keys: "index1"
           unique:  false
         - keys:  "Index2",
           unique:  false
   
   - collectionName : "mycollection1"
     create_collection : true
     shard_key      : "_id"
     default_ttl_seconds:  -1
     additional_indexes :
         - keys: "index1"
           unique:  false
         - keys:  "Index2",
           unique:  false

Terraform resource creation from yaml input file

Here is the Terraform code to create Azure Cosmos DB collections with databases based on user input in a YAML file

provider "azurerm" {
  features {}
}

data "azurerm_cosmosdb_account" "example" {
  name                = "sampledbtest"
  resource_group_name = "Venkat-RG"
}

locals {
  collections = yamldecode(file("${path.module}/config.yaml"))
}

resource "azurerm_cosmosdb_mongo_database" "database" {
  for_each = { for coll in local.collections : coll.database_name => coll }

  name                = each.key
  resource_group_name = data.azurerm_cosmosdb_account.example.resource_group_name
  account_name        = data.azurerm_cosmosdb_account.example.name
}

resource "azurerm_cosmosdb_mongo_collection" "collection" {
  for_each = { for coll in local.collections : "${coll.database_name}-${coll.collection_name}" => coll }

  name                = each.value.collection_name
  resource_group_name = data.azurerm_cosmosdb_account.example.resource_group_name
  account_name        = data.azurerm_cosmosdb_account.example.name
  database_name       = each.value.database_name

  default_ttl_seconds = each.value.default_ttl_seconds
  shard_key           = each.value.shard_key

  dynamic "index" {
    for_each = concat([{
      keys   = [each.value.shard_key, "_id"]
      unique = true
    }], each.value.indexes)
    content {
      keys   = index.value.keys
      unique = index.value.unique
    }
  }
}

config.yaml

- database_name: "tfex-cosmos-mongo-db"
  collection_name: "example-collection1"
  default_ttl_seconds: 777
  shard_key: "uniqueKey"
  indexes:
    - keys:
        - "_id"
      unique: true
- database_name: "tfex-cosmos-mongo-db1"
  collection_name: "another-collection2"
  default_ttl_seconds: 3600
  shard_key: "uniqueKey"
  indexes:
    - keys:
        - "_id"
      unique: false

Terraform apply

After executing the Terraform script, the Azure Cosmos DB collections and databases have been created in the Azure Cosmos DB account