I have an event based trigger that set on an data lake gen 2 folder. I need to trigger my pipeline only once for all these 20 files together. But now when these 20 files are loaded at a time, the event based trigger is triggering the pipeline for each file(20 times). And each trigger execution will again execute other files in the folder after the execution of triggered file.
My pipeline has foreach activity to handle all the files in my path. But my expectation is to trigger the pipeline only once regardless of the no:of files loaded.
Thank you and Sorry for the delay in response.
I was getting this incoming files from Emails(Outlook).What I did was, I created a dummy marker file from the Logic Apps.
While the Logic App moves the source files to ADLS, this maker file also will be added to the same folder. And in the Event based trigger I specified the name of the marker file instead of wildcard path. Hence it will trigger only once and Foreach can handle all the source files, before processing the source files the marker file will be deleted.
And after processing, all the source files are deleted from the incoming folder.