javamultithreadingshutdown-hook

UncaughtExceptionHandler and System.exit()

I've written a custom UncaughtExceptionHandler that should print the exception to the console and shut down the application with a custom exit code.

The class looks like this:

public class FatalUncaughtExceptionHandler implements Thread.UncaughtExceptionHandler {

    @Override
    public void uncaughtException(final Thread t, final Throwable e) {
        System.out.println("Handled exception in " + t.getName() + ":");

        e.printStackTrace();

        System.exit(ExitCodes.UNKNOWN_EXCEPTION);
    }
}

I set the UncaughtExceptionHandler in my Main.class like this:

Thread.setDefaultUncaughtExceptionHandler(new FatalUncaughtExceptionHandler());

Then I generate and start 4 threads.

In one of the running threads I purposely generate a NumberFormatException using Integer.valueOf("Test") in order to test my Handler. This works fine; Here's the output:

Handled exception in WatchdogThread:
java.lang.NumberFormatException: For input string: "Test"
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
    at java.lang.Integer.parseInt(Integer.java:580)
    at java.lang.Integer.valueOf(Integer.java:766)
    at com.csg.gfms.gms.ctmgate.runnable.WatchdogThread.run(WatchdogThread.java:43)

Now I have a problem. For some reason the thread in which the exception was thrown is not being shutdown by the System.exit() command. Apparently my ShutdownHook has a lock on it. (As seen in the output of jvisualvm):


"WatchdogThread" #38 prio=5 os_prio=0 tid=0x000000001efa3800 nid=0xd40 in Object.wait() [0x0000000021a5e000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x000000076e30a7c0> (a com.csg.gfms.gms.ctmgate.runnable.CTMShutdownHook)
        at java.lang.Thread.join(Thread.java:1252)
        - locked <0x000000076e30a7c0> (a com.csg.gfms.gms.ctmgate.runnable.CTMShutdownHook)
        at java.lang.Thread.join(Thread.java:1326)
        at java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:107)
        at java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:46)
        at java.lang.Shutdown.runHooks(Shutdown.java:123)
        at java.lang.Shutdown.sequence(Shutdown.java:167)
        at java.lang.Shutdown.exit(Shutdown.java:212)
        - locked <0x00000006c9605b00> (a java.lang.Class for java.lang.Shutdown)
        at java.lang.Runtime.exit(Runtime.java:109)
        at java.lang.System.exit(System.java:971)
        at com.csg.gfms.gms.ctmgate.handlers.FatalUncaughtExceptionHandler.uncaughtException(FatalUncaughtExceptionHandler.java:13)
        at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:1057)
        at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:1052)
        at java.lang.Thread.dispatchUncaughtException(Thread.java:1959

Even IntelliJ tells me that the System.exit command will fail. It displays a little badge next to it saying "Method will fail" when debugging my UncaughtExceptionHandler.

This leads me to my question:

Is it not allowed to call System.exit() from an UncaughtExceptionHandler?

Is the shutdown hook initiated twice in my case?

What could be the reason for the lock on the shutdown hook?

Solution

  • See that com.csg.gfms stuff in the trace?

    It's not java; it's you. That's your code that's blocking in another shutdown hook; one that is calling Thread.join.

    Generally when running into such weirdness, if it is at all possible to make a stand-alone super simple test case, then you should do so. I have done this for you:

    class Test {
        public static void main(String[] args) throws Exception {
                Thread.setDefaultUncaughtExceptionHandler(new Thread.UncaughtExceptionHandler() {
                        public void uncaughtException(Thread t, Throwable e) {
                                System.out.println("EXITING");
                                System.exit(1);
                        }
                });
                for (int i = 0; i < 10; i++) {
                        Thread t = new Thread(new Runnable() {
                                public void run() {
                                        try {
                                                Thread.sleep(1000L);
                                        } catch (Exception ignore) {}
                                        throw new RuntimeException();
                                }
                        });
                        t.start();
                }
                Thread.sleep(2000L);
                System.out.println("Still alive?");
        }
}

    When I run this, I get an arbitrary number of EXITING prints (2 to 3, it's dependent on how many cores are simultaneously working on these threads), and then the VM hard-exits. Still alive? is never printed, no locking occurs, the VM actually exits.

    Thus proving that calling System.exit() from within the uncaught exception handler is not an issue.

    The shutdown hook is not invoked twice; the shutdown hook is invoked due to you invoking System.exit, not because we got to the uncaught exception handler. But, if you're worried about this, hey, it's your app, print something in your shutdown hooks to be sure.

    The lock issue is not on the shutdown hook. You can register any amount of shutdown hooks. It's in a shutdown hook. Specifically: somebody registered an instance of com.csg.gfms.gms.ctmgate.runnable.CTMShutdownHook, that code is joining some thread, and that thread is not shutting down, thus that hook never exits, thus System.exit is not exiting the VM. The solution is to fix CTMShutdownHook, which is broken.

    Joining a thread in a shutdown hook is... well, I'll just say it bluntly: Stupid. I don't quite know what this is trying to accomplish, but the only thing I can think of is forced adherence to a bad standard. Therefore, I can foresee that you, or the author of CTMShutdownHook, first needs some introspective on how to deal with JVM shutdowns, so that they understand that the idea underlying their implementation is fundamentally misguided and needs to be rethought.

    I will do that here.

    There is this mindset that to 'properly' shut down a VM, one should never invoke System.exit, one should instead carefully tell all running threads to stop, and one should carefully manage the daemon flag on all threads, so that the VM will end up shutting down on its own volition because all still alive threads have the daemon flag set. The argument being that this gives each thread the chance to 'shut down nicely'.

    This is bad code style.

    Your app will just shut down if someone hits CTRL+C or otherwise asks the VM to exit, this will not result in a nice 'ask all threads to clean up and stop' process. In fact, your app gets zero opportunity to clean up anything if someone trips over a powercable, the computer hard-crashes, or someone terminates the app.

    This leads to the following rules:

    By adding a shutdownhook that 'joins' a thread (joining = pause this thread until that thread exits), you've created a very silly scenario where of the 3 different ways to shut an app down:

    What 'join this thread in a shutdown hook' does is effectively downgrade that second form to the (bad) third form.

    With that context, you can now fix the broken code in CTMShutdownHook, or talk to the developer of that hook and explain to them that the elegant-sounding idea of allowing all running threads to shut down nicely is in fact bad.

    Then as a more general point of principle, shutdown hooks should block as little as possible and should definitely not wait for other threads to act.