I have a problem and it's driving me crazy. The company i work for wants to offer a Kubernetes Cluster in the Azure Marketplace. I have build the necessary CNAB-Bundle and created the offering/plan in the Partner Center. The Bundle was verified by microsoft and is live inside the marketplace.
The Problem is now, that when i want to deploy the app from the marketplace to test it, the deployment fails when it tries to create the Microsoft.KubernetesConfiguration extensions.
The error message it spits out:
{
"code": "InvalidResourceType",
"message": "Resource type 'extensions' of provider namespace 'Microsoft.KubernetesConfiguration' was not found in global location for api version '2024-11-01'."
}
First i was confused because the Bundle and all the templates went through all the validation steps and there was never a Problem. So i tried to validate it again with ARM-TTK and sure enough, a similar warning popped up:
apiVersions Should Be Recent [?] apiVersions Should Be Recent (756 ms) The apiVersion 2024-11-01 was not found for the resource type: Microsoft.KubernetesConfiguration/extensions
so i tried to change the version until i found one (2023-05-01) that was validated by ARM-TTK:
apiVersions Should Be Recent [+] apiVersions Should Be Recent (340 ms)
I rebuilt the Bundle and published it again. It went through the Microsoft validation and it went live again.
But i still got the same Error:
{
"code": "InvalidResourceType",
"message": "Resource type 'extensions' of provider namespace 'Microsoft.KubernetesConfiguration' was not found in global location for api version '2023-05-01'."
}
This is the ARM-Template "clusterARMTemplate.json" i used:
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"variables": {
"plan-name": "myPlanName",
"plan-publisher": "myCompanyName",
"plan-offerID": "planOfferID",
"releaseTrain": "stable",
"clusterExtensionTypeName": "myClusterExtensionTypeName"
},
"plan": {
"name": "myPlanName",
"publisher": "myCompanyName",
"product": "productName"
},
"parameters": {
"location": {
"type": "string",
"metadata": {
"description": "Bereitstellungsregion"
}
},
"adminUsername": {
"type": "string",
"defaultValue": "azureuser",
"metadata": {
"description": "Administrator-Username für den Linux-Knoten"
}
},
"clusterName": {
"type": "string",
"defaultValue": "myApp-aks",
"metadata": {
"description": "Name des AKS Clusters"
}
},
"clusterDnsPrefix": {
"type": "string",
"defaultValue": "myAppdns",
"metadata": {
"description": "DNS Prefix für den AKS Cluster"
}
},
"agentCount": {
"type": "int",
"defaultValue": 3,
"metadata": {
"description": "Anzahl der Knoten im AKS Cluster"
}
},
"agentVMSize": {
"type": "string",
"defaultValue": "Standard_DS2_v2",
"metadata": {
"description": "VM-Größe für die AKS Knoten"
}
},
"extensionName": {
"type": "string",
"defaultValue": "myClusterExtensionTypeName",
"metadata": {
"description": "Name der Kubernetes Erweiterung"
}
},
"configurationSettings": {
"type": "object",
"metadata": {
"description": "Konfigurationseinstellungen aus der UI-Definition (Namespace, Replica Count, etc.)"
}
},
"sshPublicKey": {
"type": "string",
"metadata": {
"description": "SSH Public Key für den Zugriff auf den AKS Linux-Knoten"
}
}
},
"resources": [
{
"type": "Microsoft.ContainerService/managedClusters",
"apiVersion": "2023-10-01",
"name": "[parameters('clusterName')]",
"location": "[parameters('location')]",
"identity": {
"type": "SystemAssigned"
},
"properties": {
"dnsPrefix": "[parameters('clusterDnsPrefix')]",
"agentPoolProfiles": [
{
"name": "agentpool",
"count": "[parameters('agentCount')]",
"vmSize": "[parameters('agentVMSize')]",
"osType": "Linux",
"type": "VirtualMachineScaleSets",
"mode": "System"
}
],
"linuxProfile": {
"adminUsername": "[parameters('adminUsername')]",
"ssh": {
"publicKeys": [
{
"keyData": "[parameters('sshPublicKey')]"
}
]
}
}
}
},
{
"type": "Microsoft.KubernetesConfiguration/extensions",
"apiVersion": "2023-05-01",
"name": "[parameters('extensionName')]",
"dependsOn": [
"[resourceId('Microsoft.ContainerService/managedClusters', parameters('clusterName'))]"
],
"identity": {
"type": "SystemAssigned"
},
"plan": {
"name": "[variables('plan-name')]",
"publisher": "[variables('plan-publisher')]",
"product": "[variables('plan-offerID')]"
},
"properties": {
"extensionType": "[variables('clusterExtensionTypeName')]",
"autoUpgradeMinorVersion": true,
"configurationSettings": "[parameters('configurationSettings')]",
"scope": {
"cluster": {
"releaseNamespace": "[parameters('clusterDnsPrefix')]"
}
}
}
}
],
"outputs": {
"plan-name": {
"type": "string",
"value": "[variables('plan-name')]"
},
"plan-publisher": {
"type": "string",
"value": "[variables('plan-publisher')]"
},
"plan-offerID": {
"type": "string",
"value": "[variables('plan-offerID')]"
},
"releaseTrain": {
"type": "string",
"value": "[variables('releaseTrain')]"
},
"clusterExtensionTypeName": {
"type": "string",
"value": "[variables('clusterExtensionTypeName')]"
}
}
}
I searched the Internet for a solution but the ones i found didn't work. I saw someone mentioning that i need to register the feature in my companies azure subscription with:
az feature registration create --namespace Microsoft.KubernetesConfiguration --name ExtensionTypes
But it already was registered.
This is the first time i'm doing something like this and i can't find out what the Problem is. I hope someone here can help me with this.
First when you develop ARM templates that you will use for Marketplace item it is best to first test the deployment on your own before publishing it. Additionally I would recommend to develop via Azure Bicep, compile the template to ARM template and after that incorporate the necessary changes for being able to publish that template to the Azure Marketplace. Azure Bicep can catch more things that are not correct before the deployment compared to developing it as ARM template. The issue you have is with the scope. Resource type Microsoft.KubernetesConfiguration/extensions is extension resource type. These type of resources cannot exists on their own. They can only be deployed if they are scoped to another resource. In this case they have to be on top of Microsoft.ContainerService/managedClusters resource type. So the template needs to have that scope:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"variables": {
"plan-name": "myPlanName",
"plan-publisher": "myCompanyName",
"plan-offerID": "planOfferID",
"releaseTrain": "stable",
"clusterExtensionTypeName": "myClusterExtensionTypeName"
},
"plan": {
"name": "myPlanName",
"publisher": "myCompanyName",
"product": "productName"
},
"parameters": {
"location": {
"type": "string",
"metadata": {
"description": "Bereitstellungsregion"
}
},
"adminUsername": {
"type": "string",
"defaultValue": "azureuser",
"metadata": {
"description": "Administrator-Username für den Linux-Knoten"
}
},
"clusterName": {
"type": "string",
"defaultValue": "myApp-aks",
"metadata": {
"description": "Name des AKS Clusters"
}
},
"clusterDnsPrefix": {
"type": "string",
"defaultValue": "myAppdns",
"metadata": {
"description": "DNS Prefix für den AKS Cluster"
}
},
"agentCount": {
"type": "int",
"defaultValue": 3,
"metadata": {
"description": "Anzahl der Knoten im AKS Cluster"
}
},
"agentVMSize": {
"type": "string",
"defaultValue": "Standard_DS2_v2",
"metadata": {
"description": "VM-Größe für die AKS Knoten"
}
},
"extensionName": {
"type": "string",
"defaultValue": "myClusterExtensionTypeName",
"metadata": {
"description": "Name der Kubernetes Erweiterung"
}
},
"configurationSettings": {
"type": "object",
"metadata": {
"description": "Konfigurationseinstellungen aus der UI-Definition (Namespace, Replica Count, etc.)"
}
},
"sshPublicKey": {
"type": "string",
"metadata": {
"description": "SSH Public Key für den Zugriff auf den AKS Linux-Knoten"
}
}
},
"resources": [
{
"type": "Microsoft.ContainerService/managedClusters",
"apiVersion": "2023-10-01",
"name": "[parameters('clusterName')]",
"location": "[parameters('location')]",
"identity": {
"type": "SystemAssigned"
},
"properties": {
"dnsPrefix": "[parameters('clusterDnsPrefix')]",
"agentPoolProfiles": [
{
"name": "agentpool",
"count": "[parameters('agentCount')]",
"vmSize": "[parameters('agentVMSize')]",
"osType": "Linux",
"type": "VirtualMachineScaleSets",
"mode": "System"
}
],
"linuxProfile": {
"adminUsername": "[parameters('adminUsername')]",
"ssh": {
"publicKeys": [
{
"keyData": "[parameters('sshPublicKey')]"
}
]
}
}
}
},
{
"type": "Microsoft.KubernetesConfiguration/extensions",
"apiVersion": "2023-05-01",
"name": "[parameters('extensionName')]",
"scope": "[format('Microsoft.ContainerService/managedClusters/{0}', parameters('clusterName'))]",
"dependsOn": [
"[resourceId('Microsoft.ContainerService/managedClusters', parameters('clusterName'))]"
],
"identity": {
"type": "SystemAssigned"
},
"plan": {
"name": "[variables('plan-name')]",
"publisher": "[variables('plan-publisher')]",
"product": "[variables('plan-offerID')]"
},
"properties": {
"extensionType": "[variables('clusterExtensionTypeName')]",
"autoUpgradeMinorVersion": true,
"configurationSettings": "[parameters('configurationSettings')]",
"scope": {
"cluster": {
"releaseNamespace": "[parameters('clusterDnsPrefix')]"
}
}
}
}
],
"outputs": {
"plan-name": {
"type": "string",
"value": "[variables('plan-name')]"
},
"plan-publisher": {
"type": "string",
"value": "[variables('plan-publisher')]"
},
"plan-offerID": {
"type": "string",
"value": "[variables('plan-offerID')]"
},
"releaseTrain": {
"type": "string",
"value": "[variables('releaseTrain')]"
},
"clusterExtensionTypeName": {
"type": "string",
"value": "[variables('clusterExtensionTypeName')]"
}
}
}