shell_exec() Does Not return immediately When Executing a nohup in Background

I'm executing an exec using nohup in the background using the PHP shell_exec function. It looks like it is waiting for the executed command to finish instead of returning immediately.

private function processSystemData(array $files, string $jar_name, bool $no_server, string $out_filename): void
  {
    $i = 1;
    chdir('/home/' . Constants::ASSET_USER);
    foreach ($files as $file) {
      $filename = $file->getTempName();
      $cmd = 'nohup ' . Constants::JAVA
        . ' -jar ' . Constants::JAVA_APPS . "$jar_name.jar"
        . ' -f=' . $filename;
      $password = false;
      if (in_array($jar_name, ['netstat-query', 'server-query', 'software-versions'])) {
        $password = (new BNCPasswordDecryptor('asset'))->decryptedPassword();
        if ($no_server)
          $cmd .= ' --no-server';
        $cmd .= ' -u=' . Constants::ASSET_USER;
        $cmd .= ' -p=' . "\"" . $password . "\"";
      }
      $output_dir = '/home/' . Constants::ASSET_USER . '/system-data/';
      $outpath = $output_dir . pathinfo($out_filename, PATHINFO_FILENAME) . "-$i."
        . pathinfo($out_filename, PATHINFO_EXTENSION);
      $cmd .= ' --output=' . $outpath;
      $cmd .= ' 2>&1 &';
      $safe_cmd = $cmd;
      if ($password)
        $safe_cmd = str_replace($password, '****', $safe_cmd);
      log_message('debug', __METHOD__ . " Process $jar_name-$i started: " . $safe_cmd);
      shell_exec($cmd);
      log_message('debug', __METHOD__ . " Process $jar_name-$i ended");
      ++$i;
    } // foreach
  }

Here is the log:

INFO - 2024-05-13 10:10:01 --> CSRF token verified.
DEBUG - 2024-05-13 10:10:01 --> App\Controllers\AssetManager::processSystemData Process software-versions-1 started: nohup ~/jdk-21.0.2/bin/java -jar ~/software-versions.jar -f=/tmp/phpHZHUPF --no-server -u=info001sec -p="****" --output=/home/info001sec/system-data/software-versions-20240513_10_10_01-1.csv 2>&1 &
DEBUG - 2024-05-13 10:14:03 --> App\Controllers\AssetManager::processSystemData Process software-versions-1 ended
INFO - 2024-05-13 10:14:04 --> Session: Class initialized using 'CodeIgniter\Session\Handlers\FileHandler' driver.

What is causing it to not return immediately after the shell_exec() command? (It waited four minutes for the java code to execute)

I used the alternative PHP exec() command. That didn't make any difference.

Solution

First, let's come up with a minimal reproducible example:

shell_exec('sleep 10 &'); # pauses for 10 seconds

This causes PHP to sleep for 10 seconds - although the sleep command has returned to the shell straight away, PHP isn't terminating the shell early.

As noted in the manual page for exec but not currently for shell_exec, this is apparently because PHP is waiting for the command to close its output stream. Redirecting output to /dev/null returns immediately:

shell_exec('sleep 10 >/dev/null &'); # instant

But add in nohup and it stops working:

shell_exec('nohup sleep 10 >/dev/null &'); # pauses for 10 seconds

The problem is we need to take care of stderr as well as stdout. The order of redirections matters here!

shell_exec('nohup sleep 10 2>&1 >/dev/null &'); # pauses for 10 seconds
shell_exec('nohup sleep 10 >/dev/null 2>&1 &'); # instant! :)

If you need the output or error stream, just redirect one or both to a real file:

shell_exec('nohup sleep 10 >/tmp/combined-output 2>&1 &'); # instant
shell_exec('nohup sleep 10 >/tmp/only-stdout 2>/dev/null &'); # instant
shell_exec('nohup sleep 10 >/dev/null 2>/tmp/only-stderr &'); # instant