Why does Magento keep only the first 1000 products in a category, after saving the category?

Question

Using Magento's back-office, after saving a category which was linked to many products, only the first 1000 products (or 2000, or x000, depending on host configuration) are kept.

All other category/product links are deleted from the catalog_category_product table in the database.
This occurs even though no checkboxes have been unchecked in the products grid (Category Products tab).

No error message is displayed or logged.
Tracing data posted from the browser to the server doesn't reveal any missing product IDs, but even if you check more products on the back-office grid, when finally saving the category, the server only keeps the first x000 and removes other IDs...

Answer 1

Though some answers can be found on other websites, I thought it was worth sharing this on StackOverflow...

The source of the problem can be found in Mage_Adminhtml_Catalog_CategoryController, where the saveAction() method retrieves a big POSTed string (category_products, encoded like a query string), which is processed by PHP function parse_str():

if (isset($data['category_products']) && !$category->getProductsReadonly()) {
    $products = array();
    parse_str($data['category_products'], $products);
    $category->setPostedProducts($products);
}

Alas! As from version 5.3.9 of PHP, there is a new configuration setting called max_input_vars, which limits the number of input variables that may be accepted.
This limit is mainly applied to $_GET, $_POST and $_COOKIE superglobals, but is also used internally by the parse_str() function!
(See PHP manual)

Depending on the php.ini configuration of your host, the number of products linked to a category is therefore limited by this setting...

One solution is to increase the value of max_input_vars, in php.ini or in .htaccess:

<IfModule mod_php5.c>
    php_value max_input_vars 10000
</IfModule>

This can even be done more safely by changing this setting only for admin pages (adapt the LocationMatch pattern to your own back-office URL style):

<LocationMatch "mymagentostore/(index\.php/)?admin/">
    <IfModule mod_php5.c>
        php_value max_input_vars 10000
    </IfModule>
</LocationMatch>

(Source)

This only solves the problem until one of your categories reaches the new max number of products...

Another solution would be to fix Magento's code in order not to use the parse_str() function at this point.
For instance, in Mage_Adminhtml_Catalog_CategoryController, saveAction() method, replace:

parse_str($data['category_products'], $products);

with:

$cat_products_split = explode('&', $data['category_products']);
foreach($cat_products_split as $row) {
    $arr = array();
    parse_str($row, $arr); //This will always work
    list($k, $v) = each($arr);
    if (!empty($k) && !empty($v)) {
        $products[$k] = $v;
    }
}

(Source)

It's hard to decide which solution is the best, since the first one makes your server more vulnerable to attacks (as the max_input_vars is meant to mitigate the possibility of denial of service attacks which use hash collisions), and the second solution makes you modify a Magento core class, which can lead to further problems in future Magento upgrades...

Hope this is useful anyway, I struggled some time before I could find out why some categories kept loosing some products!