I need to deploy a solution that would process some data very infrequently (few times a year), but it requires a lot of ram and executes for around 40 minutes from a request by a user. I have looked at azure durable functions, but they have a limit of 1.5GB RAM. As I am running UMAP on the data I would need a lot more RAM. I have looked at the premium tier of the functions, but they require at least one instance to be always active, which isn't cost effective at all since this would run very infrequently? Any ideas what can I use to minimize the cost? Thanks in advance!
I tried azure durable functions, but memory limit reached.
Take a loot at Container Apps, the can scale to 0 with whichever hosting option, be that Consumption or Workload Profiles. You'll lose some features of Azure Functions (integrated Application Insights, easy configuration management) but you'll gain agility and scalability