Magento SOAP API Product List Pagination

Question

I'm trying to read the list of products from Magento over the SOAP API (V2) and try to do some/any type of pagination.

Simple scenario:

var filters = new filters();
var products = catalogProductList(out pe, Connection.Session, filters, null);

This crashes Magento with: "Allowed memory size of 1073741824 bytes exhausted (tried to allocate 72 bytes."

I've tried to add pagination by specifying two complex filters on the product_id:

filters.complex_filter = new complexFilter[]
{
    new complexFilter()
    {
        key = "product_id",
        value = new associativeEntity()
        {
            key = "gt",
            value = "400"
        }
    },
    new complexFilter()
    {
        key = "product_id",
        value = new associativeEntity()
        {
            key = "lt",
            value = "1000"
        }
    }
};

However in this scenario only the second filter is applied, the first one is ignored.

I was thinking of reading the category tree and then the assigned products but there are lots of products that are not assigned to any category or to multiple categories so I'll either miss them or get them multiple times.

Is there a way to read the products list using some type of pagination so I don't read the complete list at once? (Note: Requesting to increase memory is not really an option)

Answer 1

I've come up with a pretty good solution for this. Hope this helps someone.

$in = array();
for ($i = ($page * $size) - $size; $i < ($page * $size); $i++) {
    $in[] = $i + 1;
}
$complexFilter = array('complex_filter' => 
    array(
        array(
            'key' => 'product_id',
            'value' => array(
                'key' => 'in', 
                'value' => join(",", $in)
            )
        )
    )
);

Answer 2

It looks like the answer you need is described at https://stackoverflow.com/a/22874035/2741137

Apparently, you can capitalize PRODUCT_ID in one of the two filter conditions to work around Magento's limitation that prevents two filter conditions on the same key.