Azure databricks CI CD pipeline to delete notebooks on production
I have a CI/CD pipeline in place to deploy notebooks from dev to production in an Azure databricks workspace.
However, it is not deleting the notebooks from production, when those notebooks have been removed from development and are no longer in Azure git repository.
I want to delete all notebooks which have been removed from source, as a part of build/release process. Is there a way to achieve this?
Solution
The easiest way is when there are new commits in Azure DevOps git repository, you could redeploy the notebooks by checked the Clean Workspace Folderoption:
Otherwise, you could add a powershell script task to compare files in two folders. The follow case may give you a start: Comparing folders and content with PowerShell
- ADF Out of memory exception
- Azure DevOps pipeline is throwing a "##[error]Project file(s) matching the specified pattern were not found". when publishing my app?
- Microsoft Entra ID: Receiving Old Token Version Despite Setting accessTokenAcceptedVersion to 2
- Rule Syntax for Azure user.memberof
- (unknown) Precondition failed when trying to create MS Graph Subscription
- Azure Function App Service Bus Trigger: Process 1 message at a time
- Azure HTTP Trigger - How to extract query parameter value from input binding- cosmosDB
- Python app deployment to Azure web app from pipeline not including requirement packages
- Force Azure Function using Service Bus Queues to only process one mesage at a time
- Azure Web Role with Transient Fault Handling Block exception: The path is too long after being fully qualified
- Azure Bicep - How to create diagnostic settings for NetworkInterface of private endpoint
- Is there a way to get static ip address for Azure Data Factory
- azure key vault - lag on secret changes
- How to create connection to create - queue item - HTTP Trigger - local - Azurite - Azure Queue Storage
- Getting error "404 The specified blob does not exist" when downlading blob using Uri
- How scopes are added to token in Azure Entra ID?
- Why does my Azure Cosmos DB SQL API Container Refuse Multiple Items With Same Partition Key Value?
- Postgres on Azure kubernetes volume permission error
- Azure: Subscribing to an external MQTT Broker
- azure.storage.blob._shared.authentication.AzureSigningError: Incorrect padding - Argo workflow
- Bash variable in JMESPath query expression
- Azure Devops pipeline failing due to AZ.Accounts Module update by MSFT
- Is it possible to use email name other than "DoNotReply" in Azure Email Communication Services?
- AADSTS65001: The user or administrator has not consented to use the application with ID '<application ID>
- Access Package Endpoint doesn't seem to be working
- install docker bash script doesn't work when used as Custom Data for Azure VM
- Docker image is not updating on Azure container registry via gitlab
- Terraform - How to find Azure Kubernetes AKS vnet ID for network peering
- Using Azure DataLakeServiceClient to download a file got forbidden response after few minutes success
- Verify that Bicep can return values from Azure Key vault