terraformazure-synapseterraform-provider-azureendpointlinked-service

Unable to correctly setup Linked Service in Azure Synapse Analytics with MSI in managed Vnet


Currently I am trying to deploy some infrastructure in Azure using Terraform. So far everything worked well. Yet, when trying to deploy my Linked Services in Synapse Analytics to a Key Vault and an Azure Blob Storage (general purpose v2) they seem to work but there is one problem: Synapse is deployed with a managed Vnet and the Linked Services created with Terraform do not use the private endpoints unlike the Linked Services that are created on default by Synapse.

resource "azurerm_synapse_linked_service" "blob_ls" {
  name                 = "BLOBLinkedService${var.env_prefix}"
  synapse_workspace_id = azurerm_synapse_workspace.compass-syn-ws.id
  type                 = "AzureBlobStorage"
  type_properties_json = <<JSON
{
  "serviceEndpoint": "${azurerm_storage_account.compass_storage_account_blob.primary_blob_endpoint}",
  "accountKind": "StorageV2"
}
JSON
  depends_on = [azurerm_storage_account.compass_storage_account_blob, azurerm_synapse_workspace.compass-syn-ws, azapi_update_resource.approval_storage_blob]
}

My current best guess is, that it does not work because Terraforms' Synapse Linked Service code block does not have an option to specify that I want to make use of Synapse's Managed Identity when creating the additional Linked Services on deployment. If I am wrong on that one please tell me. Yet, I assume that to be correct because when I manually recreate the Linked Services outside of Terraform with the Managed Identity of Synapse instead of the Account Key the Linked Service is using the managed private endpoint as intended.

What are possible solutions to the problem? I have spend the whole day searching for an answer. Any help is appreciated.

I have tried to modify the JSON structure via my Terraform deployment by trying to add the relevant information to the Linked Service JSON strucuture via the type_properties_json keyword of the Synapse Linked Service codeblock. That did not work, yet I wasn't hopeful that it did as i understand it to only affect the type.properties part of the JSON structure of the Linked Service definition. Further, I am thinking about using the AzAPI provider to make manual changes to the relevant JSON strucuture of the linked service definition. Are there more elegant solutions to this problem? I cannot come up with any other solution.

I can't seem to find any official information on the topic of Linked Serivces and Managed Identity using Terraform. Neither on Hashicorps Terraform Registry nor on any open/resolved GitHub issues. Yet, the problem does not feel so niche that there shouldn't be at least a few people more searching for a solution.

Update: I have tested a bit more and it does not seem to be connected to the use of Managed Identity and Access Keys. Terraform just does not make use of the private endpoints that I have created and approved. I thought that it might be a sequencing error so I created the Linked Services at the end of the Terraform-script, when the the private endpoints are created and approved. Yet, that does not change a thing. Is there a way to force Terraform to use the private endpoints when creating the Linked Service? I can't seem to find information on that.


Solution

  • I have found a workaround myself. I removed the "faulty" Linked Service deployment and then just created it with a null_resource which contains the JSON structure which is required inside of a az command.

    resource "null_resource" "update_linked_service_blob" {
       triggers = {
         version = "${var.version_num}"
       }
       provisioner "local-exec" {
         command = <<EOT
           az synapse linked-service create --workspace-name enternamehere --name enterlsnamehere --file @${var.json_path_blob}
         EOT
       }
     }
    

    The variable for the JSON file contains the path in my terraform module:

    variable "json_path_blob" {
      default = "./json/linked_service_blob.json"
    }
    

    The correct JSON structure should look something like this (change to your storage account name):

    {
        "name": "BLOBLinkedServicedev",
        "properties": {
            "annotations": [],
            "type": "AzureBlobStorage",
            "typeProperties": {
                "serviceEndpoint": "https://<storageaccountname>.blob.core.windows.net/",
                "accountKind": "StorageV2"
            },
            "connectVia": {
                "referenceName": "AutoResolveIntegrationRuntime",
                "type": "IntegrationRuntimeReference"
            }
        }
    }