phpdoctrine-ormzend-framework2virtualboxdoctrine-mongodb

Zend Framework 2 Doctrine MongoDB ODM and Apache too slow


We're experiencing strange performance issues running:

We know for sure that this is not a DB issue (tried it with a real MongoDB instance, still same result).


Scenario

We have defined objects that work with Doctrine ODM in a manner similar to this:

<?php

namespace CatalogueManager\Document;

use Doctrine\ODM\MongoDB\Mapping\Annotations as ODM;
use Doctrine\Common\Collections\ArrayCollection;

/*
 * @ODM\Document(repositoryClass="CatalogueManager\Repository\ProductRepository")
 */
class Item
{
    /** @ODM\Id */
    protected $id;

    /** @ODM\String */
    protected $name;

    /** @ODM\Timestamp */
    protected $created;

    /** @ODM\Timestamp */
    protected $updated;

    // ---------------------------------------------------------------------- //

    /**
     * Return properties as an array. Helper method to assist with converting
     * doctrine objects to arrays so we can return front-end api calls as json.
     *
     * Required as currently Doctrine ODM do not support array hydration of
     * referenced documents.
     *
     * @access public
     * @return array
     *
     */
    public function toArray()
    {
        $arr = ['id'          => $this->id,
                'name'        => $this->name,
                'urlSlug'     => $this->urlSlug,
                'desc'        => $this->desc,
                'metaData'    => $this->metadata,
                'category'    => $this->category,
                'brand'       => $this->brand,
                'assets'      => $this->assets,
                'shipping'    => $this->shipping,
                'specs'       => $this->specs,
                'attrs'       => $this->attrs,
                'optionTypes' => $this->optionTypes
                ];

        return $arr;
    }

    // ---------------------------------------------------------------------- //

    /**
     * Getter
     *
     * @access public
     * @return string
     *
     */
    public function getId()
    {
        return $this->id;
    }

    /**
     * Getter
     *
     * @access public
     * @return string
     *
     */
    public function getName()
    {
        return $this->name;
    }

    // ---------------------------------------------------------------------- //

    /**
     * Setter
     *
     * @param string $value Property value
     *
     * @access public
     * @return void
     *
     */
    public function setName($value)
    {
        $this->name = $value;
    }

}

We're using these to import approx. 100 products into a product database. This all takes about 5 seconds on a real machine but when trying on a virtual machine, it takes approx. 25 seconds to do the same thing.

Looks like the problem could be Apache which is taking 99% load while this all is being processed, yet I'm having difficulty pinpointing what's really going on.

Any kind of advice would be appreciated...


Update

This only seems to occur when writing data. Reading data seems to be ok.

Webgrind data (screenshot) available: https://www.dropbox.com/s/jjlg7ano6epy6t1/webgrind.png?dl=0


Solution

  • After looking at some of your screenshots of XDebug data, I think you are simply using the ORM/ODM in the wrong way, since you are batch processing > 13K results.

    The correct solution for this sort of operation is explained at http://doctrine-orm.readthedocs.org/en/latest/reference/batch-processing.html

    What is happening is very simple: - the ODM loads one record from the DB - the ODM stores that record into the UnitOfWork - on flush, the ODM iterates over all entries in the UnitOfWork, looking for changed documents/entities

    If you keep storing more data in the UnitOfWork, then obviously the iteration is going to take longer times at every repetition of the operation.

    You should call ObjectManager#clear() between batch chunks being processed.