phpsymfonyshell-execpngquant

PHP shell_exec returns empty string after a while


I made a Symfony console command which uses pngquant to process and compress a long list of images. The images are read from a CSV file.

The batch is working fine until the end in local environment but in the stage environment it works for about 5 minutes and then it starts returning empty result from the shell_exec command. I even made a retry system but it's always returning empty result:

// escapeshellarg() makes this safe to use with any path
// errors are redirected to standard output
$command = sprintf(
    '%s --quality %d-%d --output %s --force %s 2>&1',
    $this->pngquantBinary,
    $minQuality,
    $maxQuality,
    escapeshellarg($tempPath),
    $path
);

// tries a few times
$data    = null;
$attempt = 0;

do {
    // command result
    $data = shell_exec($command);

    // error
    if (null !== $data) {
        $this->logger->warning('An error occurred while compressing the image with pngquant', [
            'command' => $command,
            'output'  => $data,
            'cpu'     => sys_getloadavg(),
            'attempt' => $attempt + 1,
            'sleep'   => self::SLEEP_BETWEEN_ATTEMPTS,
        ]);
        sleep(self::SLEEP_BETWEEN_ATTEMPTS);
    }

    ++$attempt;
} while ($attempt < self::MAX_NUMBER_OF_ATTEMPTS && null !== $data);

// verifies that the command has finished successfully
if (null !== $data) {
    throw new \Exception(sprintf('There was an error compressing the file with command "%s": %s.', $command, $data));
}

The problem is that the same command executed in another shell in the same environment works fine! I mean, when I log the error, if I put the exactly same command in another instance on the same server, works fine.

Even from the Symfony logs I can't read any error, where should I look for a more detailed error?

What can be causing this? Memory and processor are fine during execution!


Solution

  • After many attempts I read this question:

    Symfony2 Process component - unable to create pipe and launch a new process

    The solution was to add a call to gc_collect_cycles after flush during the loop!

    if ($flush || 0 === ($index % self::BATCH_SIZE)) {
        $this->em->flush();
        $this->em->clear();
    
        // clears the temp directory after flushing
        foreach ($this->tempImages as $tempImage) {
            unlink($tempImage);
        }
        $this->tempImages = [];
    
        // forces collection of any existing garbage cycles
        gc_collect_cycles();
    }
    

    IMPORTANT: also keep an eye on the number of disk inodes.

    df -i
    Filesystem      Inodes   IUsed  IFree IUse% Mounted on
    udev            125193     392 124801    1% /dev
    tmpfs           127004     964 126040    1% /run
    /dev/vda2      5886720 5831604  55116  100% /