javajvmrace-conditionjavaagentsjvmti

Race condition in java agent between initializing ThreadStart event callback and agent threads being created

The JVMTI documentation states the following about VMInit events.

The VM initialization event signals the completion of VM initialization.

The thread start event for the main application thread is guaranteed not to occur until after the handler for the VM initialization event returns.

And the following about ThreadStart events:

Thread start events are generated by a new thread before its initial method executes.

I'm observing an openjdk-8 java agent( specifically libjdwp.so), enabling the thread start event, starting threads with RunAgentThread, and then setting the thread start callback with SetEventCallbacks during the VMInit handler. While the main thread start event is required to occur after the VMInit handler has finished, the agent thread events are not. This means that the agent threads can trigger thread start events for which there are no callbacks. Is this intended behavior? Is the VM behavior for when a event callback is not set, but is enabled defined? If so what is it defined as?

As a side note if the agent thread waits for VMInit to complete before sending a thread start event, a deadlock occurs between VMInit waiting for the jdwp server to report a connection, and the JDWP server thread waiting for VMInit to finish.

Solution

  • JVM TI events are sent only when both conditions are met:

    1. The callback is registered with SetEventCallbacks
    2. The notifications are turned on with SetEventNotificationMode

    The order of the above calls is not important. The documentation for SetEventCallbacks says:

    An event must be enabled and have a callback in order to be sent--the order in which this function and SetEventNotificationMode are called does not affect the result.

    It's OK to have notification events enabled with no callback set. JVM does not send events in this case.

    I see no problem in jdwp agent behavior. If the agent is not interested in receiving notifications about its own threads, it can set the callback later in VMInit handler.