OpenJDK-java/lang/Throwable.fillInStackTrace - awkward implementation?

I am trying to understand the implementation of Throwable.fillInStackTrace():

private static final StackTraceElement[] UNASSIGNED_STACK = new StackTraceElement[0];
[...]
private StackTraceElement[] stackTrace = UNASSIGNED_STACK;
[...]

/**
 * Fills in the execution stack trace. This method records within this
 * {@code Throwable} object information about the current state of
 * the stack frames for the current thread.
 *
 * <p>If the stack trace of this {@code Throwable} {@linkplain
 * Throwable#Throwable(String, Throwable, boolean, boolean) is not
 * writable}, calling this method has no effect.
 *
 * @return  a reference to this {@code Throwable} instance.
 * @see     java.lang.Throwable#printStackTrace()
 */
public synchronized Throwable fillInStackTrace() {
    if (stackTrace != null ||
        backtrace != null /* Out of protocol state */ ) {
        fillInStackTrace(0);
        stackTrace = UNASSIGNED_STACK;
    }
    return this;
}

private native Throwable fillInStackTrace(int dummy);

(source OpenJDK)

I would have guess that fillInStackTrace(int) collects the stack trace from VM internals, and then creates an array of java/lang/StackTraceElement and assigns it to the Throwable.stackTrace property.

However, the standard implementation seems to clear the value of Throwable.stackTrace directly after. When running a simple sample, the information is clearly stored in the Throwable. Sample:

Throwable th = new Throwable();
th.printStackTrace(); // prints the stack up to this point

So, where is the magic / what am I misunderstanding?

PS: what does "out of protocol state" mean in the source code?

Solution

The native fillInStackTrace(int) method does not create an array of StackTraceElement instances. It captures a snapshot of the current stack trace in an implementation specific format and stores it in the backtrace variable.

Only when the array of StackTraceElement instances is requested, the method getOurStackTrace() will generate the array:

private synchronized StackTraceElement[] getOurStackTrace() {
    // Initialize stack trace field with information from
    // backtrace if this is the first call to this method
    if (stackTrace == UNASSIGNED_STACK ||
        (stackTrace == null && backtrace != null) /* Out of protocol state */) {
        int depth = getStackTraceDepth();
        stackTrace = new StackTraceElement[depth];
        for (int i=0; i < depth; i++)
            stackTrace[i] = getStackTraceElement(i);
    } else if (stackTrace == null) {
        return UNASSIGNED_STACK;
    }
    return stackTrace;
}

It’s using the helper method

native StackTraceElement getStackTraceElement(int index);

which knows how to decode the backtrace.

So if the application never accesses the stacktrace, this might be cheaper than generating the full array in advance.

You have already posted the declaration of stackTrace which initializes the field with UNASSIGNED_STACK. But there is this special constructor which allows to suppress stack traces altogether.

protected Throwable(String message, Throwable cause,
                    boolean enableSuppression,
                    boolean writableStackTrace) {
    if (writableStackTrace) {
        fillInStackTrace();
    } else {
        stackTrace = null;
    }
    detailMessage = message;
    this.cause = cause;
    if (!enableSuppression)
        suppressedExceptions = null;
}

So, normally stackTrace should be either, UNASSIGNED_STACK or an actual array created by getOurStackTrace() or set using setStackTrace(StackTraceElement[]).

If stackTrace is null, the backtrace should also be null, if stack traces are disabled. If this is not the case, the state is “out of protocol”, which means the throwable has not been generated using normal constructor invocation, but by native code.