I'm using the amazing Oban Web+Pro by @sorentwo in a Phoenix app. Previously the jobs were being executed on every node in my cluster, along with the Oban Web dashboard exposed for admins.
But now we'd like to isolate all jobs to only a few dedicated worker nodes, but keep using Oban Web along with the Phoenix app on the other web nodes to monitor job status across all nodes.
I thought it would be as simple as setting the concurrency of my queues to 0
on the web nodes:
config :my_app, Oban
queues: [
queue_1: 0,
queue_2: 0,
queue_3: 0,
]
But this just raises the following error:
** (Mix) Could not start application my_app: MyApp.Application.start(:normal, []) returned an error: shutdown: failed to start child: Oban
** (EXIT) an exception was raised:
** (ArgumentError) queue :queue_1, local_limit must be greater than 0
(oban 2.13.6) lib/oban/validation.ex:41: Oban.Validation.validate!/2
(oban 2.13.6) lib/oban/config.ex:77: Oban.Config.new/1
(oban 2.13.6) lib/oban.ex:182: Oban.start_link/1
(stdlib 4.0.1) supervisor.erl:414: :supervisor.do_start_child_i/3
(stdlib 4.0.1) supervisor.erl:400: :supervisor.do_start_child/2
(stdlib 4.0.1) supervisor.erl:384: anonymous fn/3 in :supervisor.start_children/2
(stdlib 4.0.1) supervisor.erl:1250: :supervisor.children_map/4
(stdlib 4.0.1) supervisor.erl:350: :supervisor.init_children/2
I'm probably missing a very obvious config option here so looking for help!
Thanks!
This was answered by the author of the Oban library on the version of this question crossposted to the Elixir Forum:
The solution is to set
queues: []
on the web nodes and omit the concurrency limit altogether. Or, you can use theonly
option of DynamicQueues 1 to restrict where queues run without multiple configs.