How does ForkJoinPool#awaitQuiescence actually work?

I have next implementation of RecursiveAction, single purpose of this class - is to print from 0 to 9, but from different threads, if possible:

public class MyRecursiveAction extends RecursiveAction {
    private final int num;

    public MyRecursiveAction(int num) {
        this.num = num;
    }

    @Override
    protected void compute() {
        if (num < 10) {
            System.out.println(num);
            new MyRecursiveAction(num + 1).fork();
        }
    }
}

And I thought that invoking awaitQuiescence will make current thread to wait until all tasks (submitted and forked) will be completed:

public class Main {
    public static void main(String[] args) {
        ForkJoinPool forkJoinPool = new ForkJoinPool();
        forkJoinPool.execute(new MyRecursiveAction(0));
        System.out.println(forkJoinPool.awaitQuiescence(5, TimeUnit.SECONDS) ? "tasks" : "time");
    }
}

But I don't always get correct result, instead of printing 10 times, prints from 0 to 10 times.

But if I add helpQuiesce to my implementation of RecursiveAction:

public class MyRecursiveAction extends RecursiveAction {
    private final int num;

    public MyRecursiveAction(int num) {
        this.num = num;
    }

    @Override
    protected void compute() {
        if (num < 10) {
            System.out.println(num);
            new MyRecursiveAction(num + 1).fork();
        }

        RecursiveAction.helpQuiesce();//here
    }
}

Everything works fine.

I want to know for what actually awaitQuiescence waiting?

Solution

You get an idea of what happens when you change the System.out.println(num); to System.out.println(num + " " + Thread.currentThread());

This may print something like:

0 Thread[ForkJoinPool-1-worker-3,5,main]
1 Thread[main,5,main]
tasks
2 Thread[ForkJoinPool.commonPool-worker-3,5,main]

When awaitQuiescence detects that there are pending tasks, it helps out by stealing one and executing it directly. Its documentation says:

If called by a ForkJoinTask operating in this pool, equivalent in effect to ForkJoinTask.helpQuiesce(). Otherwise, waits and/or attempts to assist performing tasks until this pool isQuiescent() or the indicated timeout elapses.

^{Emphasis added by me}

This happens here, as we can see, a task prints “main” as its executing thread. Then, the behavior of fork() is specified as:

Arranges to asynchronously execute this task in the pool the current task is running in, if applicable, or using the ForkJoinPool.commonPool() if not inForkJoinPool().

Since the main thread is not a worker thread of a ForkJoinPool, the fork() will submit the new task to the commonPool(). From that point on, the fork() invoked from a common pool’s worker thread will submit the next task to the common pool too. But awaitQuiescence invoked on the custom pool doesn’t wait for the completion of the common pool’s tasks and the JVM terminates too early.

If you’re going to say that this is a flawed API design, I wouldn’t object.

The solution is not to use awaitQuiescence for anything but the common pool¹. Normally, a RecursiveAction that splits off sub tasks should wait for their completion. Then, you can wait for the root task’s completion to wait for the completion of all associated tasks.

The second half of this answer contains an example of such a RecursiveAction implementation.

¹ awaitQuiescence is useful when you don’t have hands on the actual futures, like with a parallel stream that submits to the common pool.