Can a MassTransit JobConsumer be hosted on a different application domain (and geographical location) as the job publisher?

We are looking into MassTransit + RabbitMQ to solve a challenge we experience with automated job scheduling.

The scenario:
Our web app (OurApp), an ASP.NET Forms monolith, is deployed against multiple independent instances and with some hosted within the same infrastructure domain and some are self-hosted by our customers across multiple geographical locations on their own domain.

OurApp services multiple users varying from a few dozen to a few hundred per application instance. Each instance of OurApp has its own independent database context (SQL Server).

We have created a scheduling application (dotnet 7 + Hangfire) - let's refer to it as Automation - to schedule jobs - that we currently have in pilot.

One instance of Automation can service multiple application instances of OurApp across domains. The Hangfire schedules are configured as recurring jobs through rest API subscription endpoints exposed by Automation.

Automation then triggers these jobs via Hangfire and a SignalR connection to each of the target site and hub endpoints. This, because the code for execution has to run in OurApp within each site domain and its own database context.

Our challenge:

• Each site can have multiple schedules configured for different jobs on the same site.
• Hangfire by default will fire multiple jobs concurrently which causes some sites to experience bottlenecks at memory, processing and database resource levels.
• We would therefore want to let each site only process one job at a time.
• Each of those jobs can run for a duration of a few seconds to more than one hour at a time.

Our thought process - see diagram below:

• As the Hangfire recurring schedules triggers, let it publish its jobs as MassTransit Jobs.
  • Persist the Job Service Saga DB Context in a SQL database.
• Host the Job Consumer within the target site domain (call it OurApp.Receiver).
• The Automation MassTransit job publisher will need to target each site dynamically since more subscribers may come on board as customers adopt the service. (We already have the target site base URL from registration and SignalR hub endpoints are known.)
• The Job Consumer in turn, will then run the job via a SignalR hub connection at the target site, as that way the consumer exchange and queue can be registered at startup time by the Job Consumer host app.

I had a look at the JobConsumer.Sample from the MassTransit GitHub Repository and tried to split out the consumer part into a separate app, but I got stuck as it appears that the Job Consumer needs a connection to the saga state machine.

Is it possible to separate the Job Consumer from the Job publisher in this way, or should we look at a different approach?

I will appreciate a nudge in the right direction.

The MassTransit Job Consumer support does not have the ability to distinguish between multiple sites. The job consumers may be on separate instances, but they must be on the same broker as the job sagas to coordinate execution of the jobs.

There is no way for the job consumers to only runs jobs on a certain instance, they're load balanced across all available instances.