azureazure-data-factoryazure-storageazure-biceplinked-service

Creating a linked service with bicep file does not generate `encryptedCredential` property like the UI does


I have a bicep file that creates a linked service from a data factory to a storage account. It's pretty basic:

module Storage 'linked_services/linked_services.bicep' = {
  name: 'AzureFileStorage'
  params: {
    name: '${factoryName}/AzureFileStorage'
    referenceName: 'STORAGECONNECTION'
    storageType: 'AzureFileStorage'
    typeProperties: {
      connectionString: AzureFileStorage_connectionString
      fileShare: 'datafiles'
    }
  }
  dependsOn: [
    factoryName_STORAGECONNECTION
  ]
}

resource storageAccount 'Microsoft.Storage/storageAccounts@2023-05-01' = {
  name: storageAccountName
  location: location
  tags: {
    Application: tags.Application
    Environment: tags.Environment
  }
  sku: {
    name: 'Standard_LRS'
    tier: 'Standard'
  }
  kind: 'StorageV2'
  properties: {
    publicNetworkAccess: 'Enabled'
    minimumTlsVersion: 'TLS1_2'
    allowBlobPublicAccess: true
    allowSharedKeyAccess: true
    bypass: 'AzureServices'
    defaultAction: 'Deny'
    supportsHttpsTrafficOnly: true
    encryption: {
      services: {
        file: {
          keyType: 'Account'
          enabled: true
        }
        blob: {
          keyType: 'Account'
          enabled: true
        }
      }
      keySource: 'Microsoft.Storage'
    }
    accessTier: 'Hot'
  }
}

resource datafiles 'Microsoft.Storage/storageAccounts/fileServices/shares@2023-05-01' = {
  name: '${storageAccount.name}/default/${storageAccountName}'
  properties: {
    accessTier: 'TransactionOptimized'
    shareQuota: 5120
  }
  dependsOn: [
    storageAccount
  ]
}

linked_services.bicep

param referenceName string
param storageType string
param name string
param typeProperties object

resource linkedService 'Microsoft.DataFactory/factories/linkedServices@2018-06-01' = {
  name: name
  properties: {
    type: storageType
    connectVia: {
      referenceName: referenceName
      type: 'IntegrationRuntimeReference'
    }
    typeProperties: typeProperties
  }
}

I deployed it with ADO and it creates it but when I test the connection it doesn't work. When I go into the ADF UI and edit it manually, selecting the storage account, then the test connection works. I compared the JSON between the two and it adds the encryptedCredential property when using the UI. I'm probably going about this all wrong but I'm not sure how to get this working with a bicep file. For now I am using the account key option to authenticate. I don't have any credentials to put into a key vault or into my bicep file, nor do I want to. They just get created when I configure it in the UI. So how can I deploy my linked service with a bicep file so that it works from the start?


Solution

  • From your code, not sure how you are getting the storage connectionstring but you should be able to grab it directly form the storage you are creating:

    var AzureFileStorage_connectionString= 'DefaultEndpointsProtocol=https;AccountName=${storageAccount.name};AccountKey=${storageAccount.listKeys().keys[0].value}'
    

    Your link service module should mark typeProperties parameter as @secure to prevent seeing the connectionstring information in azure portal deployments:

    param dataFactoryName string
    param name string
    param storageType string
    @secure()
    param typeProperties object
    
    // Get a reference to data factory
    resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = {
      name: dataFactoryName
    }
    
    // create link service
    resource linkedService 'Microsoft.DataFactory/factories/linkedservices@2018-06-01' = {
      name: name
      parent: dataFactory
      properties: {
        type: storageType
        typeProperties: typeProperties
      }
    }
    

    enter image description here

    Here is full sample deployment that worked on my side

    param location string = resourceGroup().location
    param dataFactoryName string = 'df-thomastest-001'
    param storageAccountName string = 'thomastest001eaust'
    param fileShareName string = 'datafiles'
    
    // Create storage account
    resource storageAccount 'Microsoft.Storage/storageAccounts@2024-01-01' = {
      name: storageAccountName
      location: location
      sku: {
        name: 'Standard_LRS'
      }
      kind: 'StorageV2'
      properties: {
        publicNetworkAccess: 'Enabled'
        minimumTlsVersion: 'TLS1_2'
        allowBlobPublicAccess: true
        allowSharedKeyAccess: true
        accessTier: 'Hot'
      }
    }
    
    // Get a reference to the file share service
    resource fileShareService 'Microsoft.Storage/storageAccounts/fileServices@2024-01-01' existing = {
      name: 'default'
      parent: storageAccount
    }
    
    // Create file share
    resource datafiles 'Microsoft.Storage/storageAccounts/fileServices/shares@2023-05-01' = {
      name: fileShareName
      parent: fileShareService
      properties: {
        accessTier: 'TransactionOptimized'
        shareQuota: 5120
      }
    }
    
    var AzureFileStorage_connectionString = 'DefaultEndpointsProtocol=https;AccountName=${storageAccount.name};AccountKey=${storageAccount.listKeys().keys[0].value}'
    module storage 'linked_services/linked_services.bicep' = {
      name: 'AzureFileStorage'
      params: {
        dataFactoryName: dataFactoryName
        name: 'AzureFileStorage'
        storageType: 'AzureFileStorage'
        typeProperties: {
          connectionString: AzureFileStorage_connectionString
          fileShare: fileShareName
        }
      }
    }