I have an Azure App Service Plan as the following, which is supposed to have 7 GB of memory.
P2V2
420 total ACU
7 GB memory
Dv2-Series compute equivalent
83.22 USD/Month (Estimated)
I am using this to host a single App that with a single Continuous WebJob. Sure there is the website for the App itself but I do not use it at all.
Also there is only 1 instance of the App in my Service Plan so it isn't due to multiple instances sharing the total available 7 GiB memory.
The Service Plan has 2 deployment slots, 1 for Production and 1 for Staging. Both Apps (In Production and Staging) are turned off because WebJob can keep running while the App itself is off.
Here is what I see that is puzzling. From the App Service Plan overview page it says I am using around 45% of the total memory available.
But I ran into OutOfMemoryException sometimes so I started checking to see how much memory my WebJob is actually using.
It turns out that it is using 1.6 GiB out of 2 GiB available physical memory, not the 7 GiB that I was expecting.
The way I am getting the current memory is via the following code
var currentMemory = Process.GetCurrentProcess().PrivateMemorySize64;
var totalMemory = GC.GetGCMemoryInfo().TotalAvailableMemoryBytes;
According to documentation here https://learn.microsoft.com/en-us/dotnet/api/system.gcmemoryinfo.totalavailablememorybytes?view=net-6.0#System_GCMemoryInfo_TotalAvailableMemoryBytes
This property value will be the value of the COMPlus_GCHeapHardLimit environment variable, or the Server.GC.HeapHardLimit value in runtimeconfig.json, if either is set.
If the program is run in a container, this value is an implementation-defined fraction of the container's size.
Otherwise, the value of the property is the physical memory on the machine that was available for the garbage collector to use when the last garbage collection occurred.
Since I do not have environment variable, runtimeconfig.json set and it is not a container, I think GC.GetGCMemoryInfo().TotalAvailableMemoryBytes
is correctly returning total physical memory available to my WebJob on the VM, which is 2 GiB.
I also found this article thanks to Vova's comment below. https://learn.microsoft.com/en-us/azure/app-service/faq-availability-performance-application-issues#i-see-the-message--worker-process-requested-recycle-due-to--percent-memory--limit---how-do-i-address-this-issue-
It does make sense; at the same time, it seems to me the article is specific to the App Service itself. WebJob runs as a separate process from the web site. Though the symptom is the same. It seems my WebJob is JIT to 32bit rather than 64bit.
My WebJob build configuration
.NET 6 Console Application
Platform: Any CPU
Regardless, my App Service Configuration is set to x64 as shown below.
I'm trying to understand what I am doing wrong because I assume Azure is doing the right thing here.
I am using Azure App Service to run a WebJob so my goal is to maximize memory available to the WebJob with the smallest/cheapest SKU available.
I found out the problem with this is in how I run my WebJob via the run.cmd
startup script that I create in my ADO pipeline.
dotnet MyWebJob.dll
results in GC.GetGCMemoryInfo().TotalAvailableMemoryBytes
return 2 GiB as total available memory for my WeJob while my WebJob itself Process.GetCurrentProcess().PrivateMemorySize64
consumes only 1.5 GiB of memory.
Since my ADO pipeline also produces an executable specific for the platform, Windows, I also have an EXE that I could run. Start my WebJob by invoking the EXE MyWebJob.exe
results in GC.GetGCMemoryInfo().TotalAvailableMemoryBytes
return 5.5 GiB as total available memory for my WeJob while my WebJob itself Process.GetCurrentProcess().PrivateMemorySize64
bloats to 3.5 GiB of memory.
The reason for these 2 different behaviors is that the Path
environment variable contains both dotnet SDK x86 and x64, with dotnet x86 comes before dotnet x64 in the value. As a result, when I do dotnet MyWebJob.dll
, the system is using the x86 dotnet runtime, as evident in the debug console.
C:\home>dotnet --info
.NET SDK (reflecting any global.json):
Version: 6.0.100
Commit: 9e8b04bbff
Runtime Environment:
OS Name: Windows
OS Version: 10.0.14393
OS Platform: Windows
RID: win10-x86
Base Path: C:\Program Files (x86)\dotnet\sdk\6.0.100\
Host (useful for support):
Version: 6.0.0
Commit: 4822e3c3aa
I modified my ADO task that is responsible of generating the startup script for my WebJob to explicitly use the x64 version of dotnet. [01/07/2022 00:21:06 > 52cb62: INFO] C:\local\Temp\jobs\continuous\MyWebJob\n04q3opt.krt>"C:\Program Files\dotnet\dotnet.exe" MyWebJob.dll
and this fixed the issue.
The result is the same as running the EXE directly without using dotnet
as I showed above.