Update: How do i add additional group or user asigned managed identity and use users/groups names instead of UUIDs?
Main.tf
provider "azurerm" {
features {}
}
locals {
access_map = {
owner_other_access = {
permissions_access = "---"
type = "other"
},
owner_group_access = {
permissions_access = "r-x"
type = "group"
},
owner_mask_access = {
permissions_access = "rwx"
type = "mask"
},
owner_user_access = {
permissions_access = "rwx"
type = "user"
}
}
default_map = {
owner_other_default = {
permissions_default = "---"
type = "other"
},
owner_group_default = {
permissions_default = "rwx"
type = "group"
},
owner_mask_default = {
permissions_default = "rwx"
type = "mask"
},
owner_user_default = {
permissions_default = "rwx"
type = "user"
}
}
}
resource "azurerm_storage_data_lake_gen2_filesystem" "this" {
for_each = var.storage_containers
name = each.value.sc_name
storage_account_id = each.value.storage_account_id
properties = {}
dynamic "ace" {
for_each = merge(local.access_map, jsondecode(each.value.acl_access))
iterator = item
content {
type = item.value.type
scope = "access"
permissions = item.value.permissions_access
id = lookup(item.value, "id", null)
}
}
dynamic "ace" {
for_each = merge(local.default_map, jsondecode(each.value.acl_default))
iterator = item
content {
type = item.value.type
scope = "default"
permissions = item.value.permissions_default
id = lookup(item.value, "id", null)
}
}
}
teraform.tfvars
storage_containers_fs = {
raw_fs_container = {
storage_account_id = "bronze_storage_account"
sc_name = "rawfs"
acl_access = "{\"owner_group_access\":{\"permissions_access\":\"r-x\",\"type\":\"group\",\"id\":\"3f97daxxxx-xxxxxxx-xxxxx-xxxxxx-xxxxxxxxx\",\"id\":\"f5509xxxxx-xxxxxxx-xxxxx-xxxxxx-xxxxxxxxx\"}}"
acl_default = "{\"owner_group_default\":{\"permissions_default\":\"rwx\",\"type\":\"group\",\"id\":\"3f97daxxxx-xxxxxxx-xxxxx-xxxxxx-xxxxxxxxx\",\"id\":\"f5509xxxx-xxxxxxx-xxxxx-xxxxxx-xxxxxxxxx\"}}"
}
}
varaibles.tf
variable "storage_containers_fs" {
description = "Storage Containers settings"
type = map(object({
storage_account_id = string
sc_name = string
acl_access = string
acl_default = string
}))
}
My problem with this code is: When deploying for the very first time, this code works and creates a private container with ACL inside, assigning the correct rights to the two identities, in this case two groups, as seen above
When deploying this code again with a new group added to thte existing ones, the code fails by trying to replace the first group in the list with the new one added.
The new tfvars will look like this:
storage_containers_fs = {
raw_fs_container = {
storage_account_id = "bronze_storage_account"
sc_name = "rawfs"
acl_access = "{\"owner_group_access\":{\"permissions_access\":\"r-x\",\"type\":\"group\",\"id\":\"3f97daxxxx-xxxxxxx-xxxxx-xxxxxx-xxxxxxxxx\",\"id\":\"f5509xxxx-xxxxxxx-xxxxx-xxxxxx-xxxxxxxxx\",\"id\":\"deb25xxxxx-xxxxxxx-xxxxx-xxxxxx-xxxxxxxxx\"}}"
acl_default = "{\"owner_group_default\":{\"permissions_default\":\"rwx\",\"type\":\"group\",\"id\":\"3f97daxxxx-xxxxxxx-xxxxx-xxxxxx-xxxxxxxxx\",\"id\":\"f5509xxxx-xxxxxxx-xxxxx-xxxxxx-xxxxxxxxx\"}}"
}
}
This is the output from apply:
Terraform will perform the following actions:
# module.az_bronze_storage_containers_fs.azurerm_storage_data_lake_gen2_filesystem.this["raw_fs_container"] will be updated in-place
~ resource "azurerm_storage_data_lake_gen2_filesystem" "this" {
id = "https://xxxxxxxxxxxxxxxxx.dfs.core.windows.net/rawfs"
name = "rawfs"
# (4 unchanged attributes hidden)
- ace {
- permissions = "---" -> null
- scope = "access" -> null
- type = "other" -> null
}
- ace {
- permissions = "---" -> null
- scope = "default" -> null
- type = "other" -> null
}
- ace {
- permissions = "rwx" -> null
- scope = "access" -> null
- type = "mask" -> null
}
- ace {
- permissions = "rwx" -> null
- scope = "access" -> null
- type = "user" -> null
}
- ace {
- permissions = "rwx" -> null
- scope = "default" -> null
- type = "mask" -> null
}
- ace {
- permissions = "rwx" -> null
- scope = "default" -> null
- type = "user" -> null
}
- ace {
- id = "3f97xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx" -> null
- permissions = "r-x" -> null
- scope = "access" -> null
- type = "group" -> null
}
+ ace {
+ id = "deb25xxx-xxxx-xxxx-xxxx-xxxxxxxxxxx"
+ permissions = "r-x"
+ scope = "access"
+ type = "group"
}
+ ace {
+ permissions = "---"
+ scope = "access"
+ type = "other"
}
+ ace {
+ permissions = "---"
+ scope = "default"
+ type = "other"
}
+ ace {
+ permissions = "rwx"
+ scope = "access"
+ type = "mask"
}
+ ace {
+ permissions = "rwx"
+ scope = "access"
+ type = "user"
}
+ ace {
+ permissions = "rwx"
+ scope = "default"
+ type = "mask"
}
+ ace {
+ permissions = "rwx"
+ scope = "default"
+ type = "user"
}
# (1 unchanged block hidden)
}
Plan: 0 to add, 1 to change, 0 to destroy.
module.az_bronze_storage_containers_fs.azurerm_storage_data_lake_gen2_filesystem.this["raw_fs_container"]: Modifying... [id=https://xxxxxxxxxxxxxxxxx.dfs.core.windows.net/rawfs]
╷
│ Error: setting access control for root path in File System "rawfs" in Storage Account "xxxxxxxxxxxxxxxxx": datalakestore.Client#SetAccessControl: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationPermissionMismatch" Message="This request is not authorized to perform this operation using this permission.\nRequestId:e59de7e1-b01f-0066-41cd-3f5884000000\nTime:2024-01-05T11:48:36.0364932Z"
│
│ with module.az_bronze_storage_containers_fs.azurerm_storage_data_lake_gen2_filesystem.this["raw_fs_container"],
│ on ..\templates\az-storage-filesystem\main.tf line 42, in resource "azurerm_storage_data_lake_gen2_filesystem" "this":
│ 42: resource "azurerm_storage_data_lake_gen2_filesystem" "this" {
Not sure why this doesn't work.
Ideally is to add a new group without removing the existing one (first problem) AND can we make this template use group names instead of UUIDs in the tfvars (second problem)? Maybe using data.azuread_users, data.azuread_groups, data.azuread_serviceprincipals? Hardcoding IDs in the tfvars doesn't seem like a good idea. I would personally add the group name instead of the ID and let the template resolve the ID for me.
I was expecting a new identity to be added instead of replacing an existing one.
Solution
Ideally is to add a new group without removing the existing one (first problem).
To modify the existing container , you may need Storage Blob Data Owner role. Follow the Ms doc for more details
can we make this template use group names instead of
UUIDsin thetfvars(second problem)? Maybe usingdata.azuread_users,data.azuread_groups,data.azuread_serviceprincipals? Hardcoding IDs in the tfvars doesn't seem like a good idea.
If you create containers using .tfvars, it accepts an ID, not a name. However, you can use the code below to create a container without hardcoding the group ID by using a data block
terraform.tfvars
storage_containers = {
container1 = {
storage_account_id = ""
sc_name = "container1"
acl_access = "{\"owner_user_access\":{\"permissions_access\":\"rwx\",\"type\":\"user\"},\"owner_group_access\":{\"permissions_access\":\"r-x\",\"type\":\"group\",\"id\":\"data.azuread_group.example.id\"},\"owner_other_access\":{\"permissions_access\":\"---\",\"type\":\"other\"},\"owner_mask_access\":{\"permissions_access\":\"rwx\",\"type\":\"mask\"}}"
acl_default = "{\"owner_user_default\":{\"permissions_default\":\"rwx\",\"type\":\"user\"},\"owner_group_default\":{\"permissions_default\":\"rwx\",\"type\":\"group\",\"id\":\"data.azuread_group.example.id\"},\"owner_other_default\":{\"permissions_default\":\"---\",\"type\":\"other\"},\"owner_mask_default\":{\"permissions_default\":\"rwx\",\"type\":\"mask\"}}"
},
container2 = {
storage_account_id = ""
sc_name = "container2"
acl_access = "{\"owner_user_access\":{\"permissions_access\":\"rwx\",\"type\":\"user\"},\"owner_group_access\":{\"permissions_access\":\"r-x\",\"type\":\"group\",\"id\":\"Group-ID\"},\"owner_other_access\":{\"permissions_access\":\"---\",\"type\":\"other\"},\"owner_mask_access\":{\"permissions_access\":\"rwx\",\"type\":\"mask\"}}"
acl_default = "{\"owner_user_default\":{\"permissions_default\":\"rwx\",\"type\":\"user\"},\"owner_group_default\":{\"permissions_default\":\"rwx\",\"type\":\"group\",\"id\":\"Group-ID\"},\"owner_other_default\":{\"permissions_default\":\"---\",\"type\":\"other\"},\"owner_mask_default\":{\"permissions_default\":\"rwx\",\"type\":\"mask\"}}"
}
}
Here is the updated code to create an containers without .tfvars file.
provider "azurerm" {
features {}
}
locals {
access_map = {
container1 = {
permissions_access = "rwx"
type = "group"
},
container2 = {
permissions_access = "rwx"
type = "user"
}
}
default_map = {
group = {
permissions_default = "rwx"
type = "group"
},
user = {
permissions_default = "rwx"
type = "user"
},
}
}
data "azuread_group" "example" {
display_name = "Azure-AD-Group-Name"
}
data "azurerm_storage_account" "example" {
name = "storage-account-name"
resource_group_name = "RG-name"
}
resource "azurerm_storage_data_lake_gen2_filesystem" "example" {
for_each = local.access_map
name = each.key
storage_account_id = data.azurerm_storage_account.example.id
dynamic "ace" {
for_each = [data.azuread_group.example.id]
content {
id = ace.value
permissions = local.access_map[each.key].permissions_access
scope = "access"
type = local.access_map[each.key].type
}
}
dynamic "ace" {
for_each = local.default_map
content {
type = ace.key
scope = "default"
permissions = ace.value.permissions_default
id = data.azuread_group.example.id
}
}
}
Output:
- Use Terraform to assign realm-management role to service account user in Keycloak
- terraform aws aurora module: argument "engine" required, but it's actually there
- How to execute kubectl commands in HCP?
- Terraform: Create tfvars from local values
- Terraform - Upload file to S3 on every apply
- AWS ECS creates two instances
- Copy a list of terraform objects, replacing values in each object
- How to configure autoscaler in managed instance group to autoscale when required?
- Terraform output: how to get the first item from a container if any exists
- How to use if/else statement in terraform for loop
- terraform storing sensitive data in state file
- How to deal with "Error: Duplicate object key" in Terraform
- Terraform Module for multiple rules in Cloudflare ruleset
- Terraform: Import resource which is held in a module, not root
- Deploying an OpenAPI-based API gateway via Terraform?
- How to get private ip of EC2 instances spin up by ASG using Terraform
- Terraform set data source of key vault based on variable
- How to create Azure ML Registry using Terraform?
- Terraform AzureRM - route table BGP propagation failing
- Gitlab Pipeline Failing on Terraform Init
- Unable to set JSON value to variable in Terraform
- How to use output from one terraform resource for_each in another resource?
- Handle Cors from lambda to cloudfront or s3
- How to attach AWS Auto Scaling group instances to ELB Target Group in Terraform ? Should `aws_autoscaling_attachment` or `target_group_arns` be used?
- Error: Terraform Docker Image Build Fails with 403 Status Code While Using docker_image Resource
- error while applying kubernetes from yml file
- Terraform: Use count.index in variable name
- schedule autoscaler in terraform for GCP?
- Gitlab CI Terraform use plan and apply in diferent stage
- RDS Proxy: PENDING_PROXY_CAPACITY and "DBProxy Target unavailable due to an internal error"