multithreadingclojure

Can you mix agent action submission using `send` and `send-off` in Clojure? What is supposed to happen?


I understand a Clojure "agent" is a ref with an added work queue of "actions". Actions are functions that are called with the ref's value at first position and can be passed additional parameters. Actions return a new value for the ref. Thus an "agent" is a device which computes ref := (action_[n] ° action_[n-1] ° action_[n-2] ° ... ° action_[1] ° action_[0]) (ref), likely with action-induced side-effects.

A call (send-off agent action & args) results in action being enqueued for asynchronous processing by a separate thread assigned by the system to the agent (from a growable thread pool, as I understand, and once the agent's queue is empty, the thread is either stopped or goes back into the pool).

A call (send agent action & args) results in action being enqueued for asynchronous processing by thread from a fixed system pool, where the thread is switched between actions belonging to multiple agents. In that case, the action should avoid blocking or hogging the thread for all too long.

So far so good, but what happens if some actions are enqueued with send and some with send-off to the same agent? Will the system switch between its two processing modes as it dequeues the action from the agent's queue?


Solution

  • The only difference between send and send-off is which thread group is used. The scheduling of the agent and a thread is (must be) done separately.

    The only reason there are 2 thread groups in the first place is so that the send can be used for "fast running" tasks (eg in-memory/non-network operations). Tasks that are "slow" (e.g. network calls) are supposed to use send-off so they don't block the "fast" tasks.

    The definition of "fast" and "slow" is vague and up to the user. Notice the difference in language here for send vs send-off (for "potentially blocking actions" like disk or network I/O):

    https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/send


    More info here: https://clojure.org/reference/agents

    The actions of all Agents get interleaved amongst threads in a thread pool. At any point in time, at most one action for each Agent is being executed. Actions dispatched to an agent from another single agent or thread will occur in the order they were sent, potentially interleaved with actions dispatched to the same agent from other sources. send should be used for actions that are CPU limited, while send-off is appropriate for actions that may block on IO.