Magento catalog category flat index refresh Enterprise 1.13+

Not a question more of a information.

Data leek happens on the category refresh of flat table rows.

Bug reproducing steps:

create more then 1 category
set a attribute like "Default Product Listing Sort By" to a non default value for one of the categories and save it
full reindex
set index on manual
re-arrange the categories and run the enterprise reindex all cronjob

check in the flat table and see that the attribute is copied to all categories that have a position after the category that contains the attribute.

The data leek happens because the Loading of categories is done without the reset of the previous values loaded.

in the file: Enterprise\Catalog\Model\Index\Action\Category\Flat\Refresh.php

search for _reindex function(line 828)

you will see the model being loaded by $category = Mage::getModel('catalog/category');

The fix is reseting the $category variable on each loop of the foreach ($categoriesIdsChunk as $categoryId) { before the load of the new category.

Simple fix:

foreach ($categoriesIdsChunk as $categoryId) {
    if (!isset($attributesData[$categoryId])) {
        continue;
    }
    //add this line to reset the category data.
    $category = Mage::getModel('catalog/category');

    if ($category->load($categoryId)->getId()) {
        $data[] = $this->_prepareValuesToInsert(
            array_merge(
                $category->getData(),
                $attributesData[$categoryId],
                array('store_id' => $store->getId())
            )
        );
    }
 }

Solution

So, I'm going to elaborate a bit on this topic because it's important but the original topic / answer is a bit incomplete (as of Enterprise 1.14.1).

Simply rewriting a/c/c/E/Catalog/Model/Index/Action/Category/Flat/Refresh.php is only half the solution. During the drag and drop reordering of categories, there are calls to the following code in observers that watch for category save and move events...

public function processCategorySaveEvent(Varien_Event_Observer $observer)
{
    if ($this->_isLiveCategoryReindexEnabled()) {
        // ...
        $client->execute('enterprise_catalog/index_action_category_flat_refresh_row', $arguments);
    }
}

public function processCategoryMoveEvent(Varien_Event_Observer $observer)
{
    if ($this->_isLiveCategoryReindexEnabled()) {
        // ...
        $client->execute('enterprise_catalog/index_action_category_flat_refresh_changelog');
    }
};

Unfortunately, enterprise_catalog/index_action_category_flat_refresh_row and enterprise_catalog/index_action_category_flat_refresh_changelog directly extend a/c/c/E/Catalog/Model/Index/Action/Category/Flat/Refresh.php and, therefore, need to be rewritten as well.

In the end, the final fix looks something more like this...

a/c/local/Namespace/Modulename/etc/config.xml

<config>
    <global>
        <models>
            <modulename>
                <class>Namespace_Modulename_Model</class>
            </modulename>
            <!-- Enterprise_Catalog_Model_Index_Action_Category_Flat_Refresh_Row -->
            <!-- Enterprise_Catalog_Model_Index_Action_Category_Flat_Refresh_Changelog -->
            <!-- Enterprise_Catalog_Model_Index_Action_Category_Flat_Refresh -->
            <enterprise_catalog>
                <rewrite>
                    <index_action_category_flat_refresh_row>Namespace_Modulename_Model_Catalog_Index_Action_Category_Flat_Refresh_Row</index_action_category_flat_refresh_row>
                    <index_action_category_flat_refresh_changelog>Namespace_Modulename_Model_Catalog_Index_Action_Category_Flat_Refresh_Changelog</index_action_category_flat_refresh_changelog>
                    <index_action_category_flat_refresh>Namespace_Modulename_Model_Catalog_Index_Action_Category_Flat_Refresh</index_action_category_flat_refresh>
                </rewrite>
            </enterprise_catalog>
        </models>
    </global>
</config>

a/c/local/Namespace/Modulename/Model/Catalog/Index/Action/Category/Flat/Refresh/Row.php

class Namespace_Module_Model_Catalog_Index_Action_Category_Flat_Refresh_Row extends Namespace_Module_Model_Catalog_Index_Action_Category_Flat_Refresh
{
    protected $_keyColumnIdValue;

    public function __construct(array $args)
    {
        parent::__construct($args);
        if (isset($args['value'])) {
            $this->_keyColumnIdValue = $args['value'];
        }
    }

    public function execute()
    {
        if (!$this->_isFlatIndexerEnabled()) {
            return $this;
        }
        $this->_validate();
        $this->_reindex(array($this->_keyColumnIdValue));
        return $this;
    }
}

a/c/local/Namespace/Modulename/Model/Catalog/Index/Action/Category/Flat/Refresh/Changelog.php

class Namespace_Module_Model_Catalog_Index_Action_Category_Flat_Refresh_CHangelog extends Namespace_Module_Model_Catalog_Index_Action_Category_Flat_Refresh
{
    public function execute()
    {
        if (!$this->_isFlatIndexerEnabled()) {
            return $this;
        }
        $this->_validate();
        $changedIds = $this->_selectChangedIds();
        if (is_array($changedIds) && count($changedIds) > 0) {
            $idsBatches = array_chunk($changedIds, Mage::helper('enterprise_index')->getBatchSize());
            foreach ($idsBatches as $changedIds) {
                $this->_reindex($changedIds);
            }
            $this->_setChangelogValid();
        }
        return $this;
    }
}

a/c/local/Namespace/Modulename/Model/Catalog/Index/Action/Category/Flat/Refresh.php

foreach ($categoriesIdsChunk as $categoryId) {
    if (!isset($attributesData[$categoryId])) {
        continue;
    }

    // Flat Reindexing Fix
    // Flat Reindexing Fix
    $category = Mage::getModel('catalog/category');
    // Flat Reindexing Fix
    // Flat Reindexing Fix

    if ($category->load($categoryId)->getId()) {
        $data[] = $this->_prepareValuesToInsert(
            array_merge(
                $category->getData(),
                $attributesData[$categoryId],
                array('store_id' => $store->getId())
            )
        );
    }
}

I feel like there might be a better way to accomplish this that's less invasive but it does work. Just be aware of this code when you're updating as it'll probably need to be attended to.