Azure Data Factory - MQ data flow
I have been tasked to create an Azure Data Factory pipeline that will process messages being generated from an MQ Farm and that are stored in Data Storage in .xml format and then ingest them in a SharePoint Table.
The question is how would your approach be in that scenario to slice the .xml files in smaller pieces? The .xml files are nesting a lot of records in one file (with a valid separator on each record) and I wish to discard some while process the valid ones.
P.S.: For receiving and storing the MQ Farm messages I am using a logic app before Azure Data Factory
Solution
OK the solution was more obvious than previously thought... Solved from the logic app designer and saving to blob
- Re-send dead letter message in Azure Bus
- How to do random sampling of rows in Synapse Analytics serverless SQL pool?
- How to skip admin approval for external users in an Entra ID multi-tenant application
- Bicep How to use external file to be used in Deployment Script?
- Insufficient Privileges error when retrieving AD User's Entra ID from Microsoft Graph API using email address
- What is the correct way to set a cookie expiration when using Azure AD to login users to an ASP.NET Core 5 Web Application?
- How can I Parse Json in a Azure Function
- Alchemy Websockets: Can't host server on Azure
- How to connect secrets value stored in Azure key vault to Azure app service env variables directly
- I need to use some query results in an alert email notification in Azure ApplicationInsights
- Azure DevOps Pull work items in tree level in Excel
- Azure Event Hubs to share specific partitions with specific consumer groups
- Azure AD B2C vs Microsoft Entra External ID
- How to query OAuth2 permission grants for service principals in Entra ID using Microsoft Graph PowerShell
- How to get the app role's value given an app role ID in Entra ID using Microsoft Graph Powershell?
- How to list all directory extension definitions within an Entra ID tenant?
- How to get error message from action in Logic App
- How do I enable my Azure pipeline to checkout a submodule using Git?
- Azure Mobile App using existing database that has identity column not called id
- Devops self hosted agent - submodule checkout SSH host key verification failed
- How can I read a XML file Azure Databricks Spark
- Azure Command-Line Interface (CLI) error running .NET 8 console app
- Azure windows VM extensions are provisioning failed state VM agent is "Ready" but Backup failed at VM running State GuestAgentSnapshotTaskStatusError
- Android scheme not accepted by IOS when submitting react-native app through Expo EAS
- Azure PowerShell command to list all Azure VM SKU Sizes enabled for Confidential Computing Azure ACC
- Access Azure DevOps Artifact Feed from different organization
- Duplicate record handling with Azure Data Explorer
- File upload fails with 413 Request Entity Too Large in Azure app service but not while running locally
- Azure Tables: best practices for choosing partition/row keys
- Access Denied when accessing Azure Key vault from Azure Functions