Update: How do i add additional group or user asigned managed identity and use users/groups names instead of UUIDs?
Main.tf
provider "azurerm" {
features {}
}
locals {
access_map = {
owner_other_access = {
permissions_access = "---"
type = "other"
},
owner_group_access = {
permissions_access = "r-x"
type = "group"
},
owner_mask_access = {
permissions_access = "rwx"
type = "mask"
},
owner_user_access = {
permissions_access = "rwx"
type = "user"
}
}
default_map = {
owner_other_default = {
permissions_default = "---"
type = "other"
},
owner_group_default = {
permissions_default = "rwx"
type = "group"
},
owner_mask_default = {
permissions_default = "rwx"
type = "mask"
},
owner_user_default = {
permissions_default = "rwx"
type = "user"
}
}
}
resource "azurerm_storage_data_lake_gen2_filesystem" "this" {
for_each = var.storage_containers
name = each.value.sc_name
storage_account_id = each.value.storage_account_id
properties = {}
dynamic "ace" {
for_each = merge(local.access_map, jsondecode(each.value.acl_access))
iterator = item
content {
type = item.value.type
scope = "access"
permissions = item.value.permissions_access
id = lookup(item.value, "id", null)
}
}
dynamic "ace" {
for_each = merge(local.default_map, jsondecode(each.value.acl_default))
iterator = item
content {
type = item.value.type
scope = "default"
permissions = item.value.permissions_default
id = lookup(item.value, "id", null)
}
}
}
teraform.tfvars
storage_containers_fs = {
raw_fs_container = {
storage_account_id = "bronze_storage_account"
sc_name = "rawfs"
acl_access = "{\"owner_group_access\":{\"permissions_access\":\"r-x\",\"type\":\"group\",\"id\":\"3f97daxxxx-xxxxxxx-xxxxx-xxxxxx-xxxxxxxxx\",\"id\":\"f5509xxxxx-xxxxxxx-xxxxx-xxxxxx-xxxxxxxxx\"}}"
acl_default = "{\"owner_group_default\":{\"permissions_default\":\"rwx\",\"type\":\"group\",\"id\":\"3f97daxxxx-xxxxxxx-xxxxx-xxxxxx-xxxxxxxxx\",\"id\":\"f5509xxxx-xxxxxxx-xxxxx-xxxxxx-xxxxxxxxx\"}}"
}
}
varaibles.tf
variable "storage_containers_fs" {
description = "Storage Containers settings"
type = map(object({
storage_account_id = string
sc_name = string
acl_access = string
acl_default = string
}))
}
My problem with this code is: When deploying for the very first time, this code works and creates a private container with ACL inside, assigning the correct rights to the two identities, in this case two groups, as seen above
When deploying this code again with a new group added to thte existing ones, the code fails by trying to replace the first group in the list with the new one added.
The new tfvars will look like this:
storage_containers_fs = {
raw_fs_container = {
storage_account_id = "bronze_storage_account"
sc_name = "rawfs"
acl_access = "{\"owner_group_access\":{\"permissions_access\":\"r-x\",\"type\":\"group\",\"id\":\"3f97daxxxx-xxxxxxx-xxxxx-xxxxxx-xxxxxxxxx\",\"id\":\"f5509xxxx-xxxxxxx-xxxxx-xxxxxx-xxxxxxxxx\",\"id\":\"deb25xxxxx-xxxxxxx-xxxxx-xxxxxx-xxxxxxxxx\"}}"
acl_default = "{\"owner_group_default\":{\"permissions_default\":\"rwx\",\"type\":\"group\",\"id\":\"3f97daxxxx-xxxxxxx-xxxxx-xxxxxx-xxxxxxxxx\",\"id\":\"f5509xxxx-xxxxxxx-xxxxx-xxxxxx-xxxxxxxxx\"}}"
}
}
This is the output from apply:
Terraform will perform the following actions:
# module.az_bronze_storage_containers_fs.azurerm_storage_data_lake_gen2_filesystem.this["raw_fs_container"] will be updated in-place
~ resource "azurerm_storage_data_lake_gen2_filesystem" "this" {
id = "https://xxxxxxxxxxxxxxxxx.dfs.core.windows.net/rawfs"
name = "rawfs"
# (4 unchanged attributes hidden)
- ace {
- permissions = "---" -> null
- scope = "access" -> null
- type = "other" -> null
}
- ace {
- permissions = "---" -> null
- scope = "default" -> null
- type = "other" -> null
}
- ace {
- permissions = "rwx" -> null
- scope = "access" -> null
- type = "mask" -> null
}
- ace {
- permissions = "rwx" -> null
- scope = "access" -> null
- type = "user" -> null
}
- ace {
- permissions = "rwx" -> null
- scope = "default" -> null
- type = "mask" -> null
}
- ace {
- permissions = "rwx" -> null
- scope = "default" -> null
- type = "user" -> null
}
- ace {
- id = "3f97xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx" -> null
- permissions = "r-x" -> null
- scope = "access" -> null
- type = "group" -> null
}
+ ace {
+ id = "deb25xxx-xxxx-xxxx-xxxx-xxxxxxxxxxx"
+ permissions = "r-x"
+ scope = "access"
+ type = "group"
}
+ ace {
+ permissions = "---"
+ scope = "access"
+ type = "other"
}
+ ace {
+ permissions = "---"
+ scope = "default"
+ type = "other"
}
+ ace {
+ permissions = "rwx"
+ scope = "access"
+ type = "mask"
}
+ ace {
+ permissions = "rwx"
+ scope = "access"
+ type = "user"
}
+ ace {
+ permissions = "rwx"
+ scope = "default"
+ type = "mask"
}
+ ace {
+ permissions = "rwx"
+ scope = "default"
+ type = "user"
}
# (1 unchanged block hidden)
}
Plan: 0 to add, 1 to change, 0 to destroy.
module.az_bronze_storage_containers_fs.azurerm_storage_data_lake_gen2_filesystem.this["raw_fs_container"]: Modifying... [id=https://xxxxxxxxxxxxxxxxx.dfs.core.windows.net/rawfs]
╷
│ Error: setting access control for root path in File System "rawfs" in Storage Account "xxxxxxxxxxxxxxxxx": datalakestore.Client#SetAccessControl: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationPermissionMismatch" Message="This request is not authorized to perform this operation using this permission.\nRequestId:e59de7e1-b01f-0066-41cd-3f5884000000\nTime:2024-01-05T11:48:36.0364932Z"
│
│ with module.az_bronze_storage_containers_fs.azurerm_storage_data_lake_gen2_filesystem.this["raw_fs_container"],
│ on ..\templates\az-storage-filesystem\main.tf line 42, in resource "azurerm_storage_data_lake_gen2_filesystem" "this":
│ 42: resource "azurerm_storage_data_lake_gen2_filesystem" "this" {
Not sure why this doesn't work.
Ideally is to add a new group without removing the existing one (first problem) AND can we make this template use group names instead of UUIDs in the tfvars (second problem)? Maybe using data.azuread_users, data.azuread_groups, data.azuread_serviceprincipals? Hardcoding IDs in the tfvars doesn't seem like a good idea. I would personally add the group name instead of the ID and let the template resolve the ID for me.
I was expecting a new identity to be added instead of replacing an existing one.
Solution
Ideally is to add a new group without removing the existing one (first problem).
To modify the existing container , you may need Storage Blob Data Owner role. Follow the Ms doc for more details
can we make this template use group names instead of
UUIDsin thetfvars(second problem)? Maybe usingdata.azuread_users,data.azuread_groups,data.azuread_serviceprincipals? Hardcoding IDs in the tfvars doesn't seem like a good idea.
If you create containers using .tfvars, it accepts an ID, not a name. However, you can use the code below to create a container without hardcoding the group ID by using a data block
terraform.tfvars
storage_containers = {
container1 = {
storage_account_id = ""
sc_name = "container1"
acl_access = "{\"owner_user_access\":{\"permissions_access\":\"rwx\",\"type\":\"user\"},\"owner_group_access\":{\"permissions_access\":\"r-x\",\"type\":\"group\",\"id\":\"data.azuread_group.example.id\"},\"owner_other_access\":{\"permissions_access\":\"---\",\"type\":\"other\"},\"owner_mask_access\":{\"permissions_access\":\"rwx\",\"type\":\"mask\"}}"
acl_default = "{\"owner_user_default\":{\"permissions_default\":\"rwx\",\"type\":\"user\"},\"owner_group_default\":{\"permissions_default\":\"rwx\",\"type\":\"group\",\"id\":\"data.azuread_group.example.id\"},\"owner_other_default\":{\"permissions_default\":\"---\",\"type\":\"other\"},\"owner_mask_default\":{\"permissions_default\":\"rwx\",\"type\":\"mask\"}}"
},
container2 = {
storage_account_id = ""
sc_name = "container2"
acl_access = "{\"owner_user_access\":{\"permissions_access\":\"rwx\",\"type\":\"user\"},\"owner_group_access\":{\"permissions_access\":\"r-x\",\"type\":\"group\",\"id\":\"Group-ID\"},\"owner_other_access\":{\"permissions_access\":\"---\",\"type\":\"other\"},\"owner_mask_access\":{\"permissions_access\":\"rwx\",\"type\":\"mask\"}}"
acl_default = "{\"owner_user_default\":{\"permissions_default\":\"rwx\",\"type\":\"user\"},\"owner_group_default\":{\"permissions_default\":\"rwx\",\"type\":\"group\",\"id\":\"Group-ID\"},\"owner_other_default\":{\"permissions_default\":\"---\",\"type\":\"other\"},\"owner_mask_default\":{\"permissions_default\":\"rwx\",\"type\":\"mask\"}}"
}
}
Here is the updated code to create an containers without .tfvars file.
provider "azurerm" {
features {}
}
locals {
access_map = {
container1 = {
permissions_access = "rwx"
type = "group"
},
container2 = {
permissions_access = "rwx"
type = "user"
}
}
default_map = {
group = {
permissions_default = "rwx"
type = "group"
},
user = {
permissions_default = "rwx"
type = "user"
},
}
}
data "azuread_group" "example" {
display_name = "Azure-AD-Group-Name"
}
data "azurerm_storage_account" "example" {
name = "storage-account-name"
resource_group_name = "RG-name"
}
resource "azurerm_storage_data_lake_gen2_filesystem" "example" {
for_each = local.access_map
name = each.key
storage_account_id = data.azurerm_storage_account.example.id
dynamic "ace" {
for_each = [data.azuread_group.example.id]
content {
id = ace.value
permissions = local.access_map[each.key].permissions_access
scope = "access"
type = local.access_map[each.key].type
}
}
dynamic "ace" {
for_each = local.default_map
content {
type = ace.key
scope = "default"
permissions = ace.value.permissions_default
id = data.azuread_group.example.id
}
}
}
Output:
- Ec2 instance creation using terraform assigns public IP
- Load variable values from the given .tfvars file got an error to many arguments
- Can we create a data source through a file or local variables?
- Error in assigning diff types of maps in a conditional evaluation
- How Errors can be handled in Terraform?
- The provider hashicorp/azurerm does not support resource type "azurerm_resource_group_policy_assignment"
- Terraform Plan Error: Unable to Build Authorizer for Azure Resource Manager API
- Terraform 1.9.8: Variable Validation Condition Always Evaluates True When Referencing Another Variable
- adding extra element to a list in terraform if condition is met
- Springboot lambda on localstack with terraform, docker: Error loading class StreamLambdaHandler: Metaspace","errorType":"java.lang.OutOfMemoryError
- How to organize Terraform modules for multiple environments?
- azurerm_virtual_machine doesn't remove managed storage_os_disk on destroy
- terraform use a dynamic count to get value from a list
- How to compare a variable against two strings?
- Is there an AND/OR conditional operator?
- How to output an input variable?
- In GitHub Actions, how can I run a terraform plan when a PR is opened, and run an apply after the PR is merged?
- Heredoc vs indented heredoc
- How to reuse variable *definitions* in Terraform?
- How to define global variables in terraform?
- Invalid characters in heredoc anchor
- Terraform: Using versioned modules stored in AWS CodeCommit
- Terraform: Error: retrieving static website properties for Storage Account (Subscription: *** : context deadline exceeded
- Terraform Amazon Machine Image script
- Merging maps with variables
- How can I pass credentials in Terraform?
- Access Kudu CLI via Terraform
- Unable to access variables inside a module
- Difference between var="${local.var}" and var=local.var in the context of invoking a module
- How can I remove a resource from terraform state?