I am trying to load data through snowpipes and somehow all the files are not getting loaded, these files are from different sources including automated jobs, manual uploads, data extracts. The one thing I am seeing in common is - the files that do not have md5 are not ingested automatically, while the files that do are. So wanted to confirm if anybody knows that md5 is necessary for a file to be auto-ingested?
Thanks
Turns out ADLS gen2 API needs explicit flush close in order to create a notification for snowflake which leads to auto ingest. Added this to the azure function and it seems to be working fine now