I am trying to understand two-phase-commit and it is not clear to me when each local site executes its portion of the distributed transaction.
Does that happen before the prepare messages are sent. That is does it happen before the two-phase-commit xa protocol is even run?
Or does each site executes its portion of the distributed transaction after receiving the prepare message, meaning the prepare message itself also includes the transaction query to be executed?
Yes, execution happens before prepare messages are sent. You should assume that the entire 2PC protocol runs within the commit() operation after everything has already been executed. Consider the following imagined trace of a distributed transaction that eventually commits. Indentation means procedure nesting:
transactionalOperation() is a procedure at some client site
begin() is a remote invocation to the coordinator
create global XID
return global XID
XID is added to the thread-local context at the caller
executeOperation(params) is a remote invocation to a participant site
XID is implicitly propagated to the server side thread
inform coordinator that this site is a participant
execute the operation! (acquiring locks)
... more operations ....
commit() is a remote invocation to the coordinator
XID is implicitly propagated to the server side thread
-------------- 2PC starts here ----------------
for all registered participants:
prepare(XID) is a remote invocation to each site
make sure that the transaction can be committed,
usually by writing and flushing a log
return OK
gather decisions
for all reguistered participants:
commit(XID) is a remote invocation to each site
log transaction commit
(release locks)
-------------- 2PC ends here ----------------
XID is removed from the thread-local context at the caller
transaction is complete!
Actually, as you can see, the prepare message will be sent by the coordinator only to those sites that have previously executed something within the context of such transaction, and thus have previously registered as participants in that transaction.