phplinuxmultithreadingcron

Running multiple PHP scripts at once from command line initiated by cron jobs


I have a question concerning cron jobs.

Here is a brief look at the situation:

  1. I call a php script every minute (*****) (PHP CLI).

  2. The PHP script makes a database request to check if there are any emails to send for that specific minute and sends them accordingly.

  3. The PHP script needs more than 1 minute for execution (huge database -> lots of emails to send -> takes time even though using multiple threads)

  4. The same PHP script gets called the next minute (*****), while its first execution hasn't finished its execution yet.

Will the first execution interrupt, when the script is called again (every minute it is called), even though the first execution hasn't finished yet?

I hope the situation is clear for you.

Operating System: Linux


Solution

  • Will the first execution interrupt, when the script is called again (every minute it is called), even though the first execution hasn't finished yet?

    No, it will not interrupt unless you handle this case in the script being called. A common way to solve the problem is using locks, or implementing the mutual exclusion.

    There are numerous ways to implement locking in PHP, and there is no "best" way. You can whether pick one that suits best for the back ends available on your platform, or implement multiple lockers for different platforms and maybe even different use cases. Beware, you should make sure that you are always using the same locker for specific jobs on the same host!

    Some of the popular tools that can be used as back ends for locking:

    Example

    The following code implements an abstract locker class and a sample implementation based on phpredis extension.

    namespace Acme;
    
    class Factory {
      /// @var \Redis
      private static $redis;
    
      public static function redis() {
        if (!static::$redis) {
          try {
            static::$redis = new \Redis();
            // In practice you should fetch the host from a configuration object.
            static::$redis->pconnect('/tmp/redis.sock');
          } catch (\Exception $e) {
            trigger_error($e->getMessage(), E_USER_WARNING);
            return false;
          }
        }
        return static::$redis;
      }
    
      /**
       * @param mixed $id ID of a job or group of jobs
       * @return AbstractLocker
       */
      public static function locker($id) {
        return new RedisLocker($id);
      }
    }
    
    
    abstract class AbstractLocker {
      abstract public function __construct($id);
      abstract public function lock();
      abstract public function unlock();
      abstract public function isLocked();
    }
    
    
    class RedisLocker extends AbstractLocker {
      /// Key prefix
      const PREFIX = 'lock/';
    
      /// @var \Redis
      private static $redis;
      /// @var string DB item key
      private $key;
      /// @var int Expiration time in seconds
      private $expire = 86400;
    
      /**
       * @param mixed $id ID of a job or group of jobs
       */
      public function __construct($id) {
        if (!static::$redis) {
          static::$redis = Factory::redis();
        }
        $this->key = static::PREFIX . '/' . $id;
      }
    
      public function lock() {
        $this->_fixDeadlocks();
        $r = static::$redis;
    
        // Set the key to the current process ID
        // within a transaction (see http://redis.io/topics/transactions).
        $r->multi();
        $result = $r->setnx($this->key, getmypid());
        $r->setTimeout($this->key, $this->expire);
        $r->exec();
    
        return (bool) $result;
      }
    
      public function unlock() {
        $r = static::$redis;
    
        // Delete the key from DB within a transaction.
        $r->multi();
        $result = $r->delete($this->key);
        $r->exec();
    
        return (bool) $result;
      }
    
      public function isLocked() {
        $this->_fixDeadlocks();
        return (bool) static::$redis->exists($this->key);
      }
    
      private function _fixDeadlocks() {
        $r = static::$redis;
    
        if (!$r->exists($this->key) || (!$pid = $r->get($this->key))) {
          return;
        }
    
        $running = (bool) posix_kill($pid, 0);
        if ($pid && $running) {
          // Another process is running normally
          return;
        }
    
        if (!$running) {
          // Process is not running, so the keys must not exist
          if ($r->exists($this->key) && $pid == $r->get($this->key)) {
            // Deadlock found
            $this->unlock();
          }
        }
      }
    }
    
    //////////////////////////////////////////////////////////////////
    // Usage
    
    $id = 'Bubbles';
    $locker = Factory::locker($id);
    if ($locker->isLocked()) {
      trigger_error("$id job is locked");
      exit(1);
    }
    $locker->lock();
    for ($i = 0; $i < 10; ++$i) { echo '. o O '; usleep(1e6); }
    echo PHP_EOL;
    $locker->unlock();
    

    Testing

    Terminal A

    $ php script.php
    . o O . o O . o O . o O . o O . o
    

    Terminal B

    $ php script.php
    Notice: Bubbles job is locked in /home/ruslan/tmp/script.php on line 121