javajvmstack-overflow

If one thread uses a lot of stack frames, could that cause stack overflow on an "innocent" thread?

For context, I have a situation where less-than-trusted user code needs to run on a shared system. The code is subject to various static analysis passes and quality assurances, and some ways it can fail are tolerable, so this question ONLY pertains to stack overflows.

My question is: If two threads are running in parallel on the JVM, one of which is behaving innocently (shallow stack depth) and the other is in some infinite/very deep recursion which would eventually cause an SO, is it guaranteed that the "guilty" thread will be the one to throw the exception? Or are both threads using up the same shared stack size, and the "innocent" one may just get unlucky and be the first to reach for memory that goes over the limit?

If it's the latter, is there a way to enforce the other behavior?

Solution

  • From JVMS §2.5.2,

    Each Java Virtual Machine thread has a private Java Virtual Machine stack, created at the same time as the thread.

    If the computation in a thread requires a larger Java Virtual Machine stack than is permitted, the Java Virtual Machine throws a StackOverflowError.

    The stack is not shared among all the threads. Each thread has its own stack. The "innocent thread" by definition does not use too much stack space on its own, so it will not "require a larger Java Virtual Machine stack than is permitted", so it will not throw a StackOverflowError.

    But note that:

    If Java Virtual Machine stacks can be dynamically expanded, and expansion is attempted but insufficient memory can be made available to effect the expansion, or if insufficient memory can be made available to create the initial Java Virtual Machine stack for a new thread, the Java Virtual Machine throws an OutOfMemoryError.

    It could be the case that the "innocent thread" can throw an OutOfMemoryError instead of StackOverflowError. Consider the case where the "innocent thread" calls a method, and the current stack size is not enough to store the frame. This causes the JVM to try to expand the stack (assuming the JVM implements an expanding stack), which in turn causes an OutOfMemoryError because all the memory had been taken up by the stack of the "guilty thread".