We have two deployments (Prod and Test) of Azure Webjob running with TimerTrigger. Both the web apps have single instance. According to this article, TimeTriggers use singleton locks to make sure no parallel invocation. These two instances use the same storage accounts. The issue we are facing is that only one of the deployments seem to acquire the lock while other is unable to acquire the lock. If we stop the first webjob the second acquires lock and starts processing and vice-versa.
Does the lock depend on storage accounts? How can we make sure that both these deployments have separate locking mechanism and run at the same time?
You're right, you have to use different storage accounts.
From the documentation:
Behind the scenes, TimerTrigger uses the Singleton feature of the WebJobs SDK to ensure that only a single instance of your triggered function is running at any given time. When the JobHost starts up, for each of your TimerTrigger functions a blob lease (the Singleton Lock) is taken. This distrubuted lock ensures that only a single instance of your scheduled function is running at any time. If the blob for that function is not currently leased, the function will acquire the lease and start running on schedule immediately. If the blob lease cannot be acquired, it generally means that another instance of that function is running, so the function is not started in the current host. When this happens, the host will continue to periodically check to see if it can acquire the lease. This is done as a sort of "recovery" mode to ensure that when running steady state, if an instance goes down, another instance notice and pick up where the other left off.
If you look at the implementation of the locking mecanism (StorageScheduleMonitor.cs):
So based on @volodymyr-bilyachat answer, there are two possibilities:
Having separated storage accounts: makes sense if you have a storage account per environment (dev/staging/prod)
Specifying the HosId property of the JobHostConfiguration class:
var config = new JobHostConfiguration();
config.HostId = "dev|staging|prod";