A QuickFIX java application could be started by using the
Acceptor.start() or Initiator.start() methods.
Is there a way to send a shutdown signal to the application that is running, instead of killing it?
If it is sufficient to disconnect the sessions then maybe access via JMX could be what you are looking for.
See here for example code: https://quickfixj.org/usermanual/2.3.0/usage/jmx.html
If you really want to shutdown the whole app then that has to be implemented in your own application's code.