Context:
Assume an Azure Firewall with a Firewall Policy that is deployed by pipeline PA using Terraform. In this configuration is also Rule Collection Group RCGA, i.e. RCGA is deployed by running PA.
Then there is another repository with a separate Terraform configuration deployed with pipeline PB. This pipeline is a central pipeline for deploying projects (of something that is not relevant for here). Each project has a configuration file that is passed to the pipeline and then runs the pipeline for that specific project. Also, each project has its own state file. Each project shall have its own rule collection with rules associated to RCGA.
Problem:
There should be only one firewall and a firewall can as far as I know only have one policy associated.
I found https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/firewall_network_rule_collection but these seem to be classic rules if I am not mistaken and there is also no way to associate such collections with a rule collection group.
Then there is https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/firewall_policy_rule_collection_group. The problem here is that it does not allow for decentralized deployment of rule collections (as there is no separate rule collection resource and remember there is a state file per project and that group would then be in each state project file…).
Then I thought about changing the hierarchy, i.e. that each project becomes its own rule collection group. But there is the problem that there is a limit of 90 groups per policy (https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits#azure-firewall-limits). Therefore, also no option.
I also checked the portal to see how it works there – as there you can create a new rule collection and associate it with an existing rule collection group. But it seems that this happens again via rule collections groups (PUT https://management.azure.com/subscriptions/xxx/resourceGroups/xxx/providers/Microsoft.Network/firewallPolicies/xxx/ruleCollectionGroups/xxx?api-version=2022-07-01). This also rules out the option to use azapi (at least for azapi_resource part, as the group would be again a fully managed resource in the scope of the project).
Finally I found https://learn.microsoft.com/en-us/cli/azure/network/firewall/policy/rule-collection-group/collection?view=azure-cli-latest#az-network-firewall-policy-rule-collection-group-collection-add-filter-collection what basically does what I need, but it is in preview (this alone rules it out as we talk about productive workloads) and I also don’t want to use extra scripting.
Questions:
Which approach do you recommend to achieve the decentralized management of rule collections and related rules that shall be associated with a rule collection group which resource is managed somewhere else?
Would https://registry.terraform.io/providers/Azure/azapi/latest/docs/resources/update_resource be an option? And if so, what do I need to consider when using update_resource (e.g. how this “child resource” and the related state is managed in relation to the management and related state of the group somewhere else)?
EDIT: update_resource can't be used as it replaces the full array of collections every time, i.e. it is not possible to manage the collections decentrally with this approach (at least not without getting first all, and then modifying it e.g. by appending a new collection -> not an approach I will pursue)
The decentralized approach as described above is currently not possible to my knowledge.
As alternative one approach is to manage the rule collection group centrally (but separate from the central firewall deployment) with a single state file for this group (which then contains the rule collections and rules).