How to get more search results than the server's sizelimit with Python LDAP?

Question

I am using the python-ldap module to (amongst other things) search for groups, and am running into the server's size limit and getting a SIZELIMIT_EXCEEDED exception. I have tried both synchronous and asynchronous searches and hit the problem both ways.

You are supposed to be able to work round this by setting a paging control on the search, but according to the python-ldap docs these controls are not yet implemented for search_ext(). Is there a way to do this in Python? If the python-ldap library does not support it, is there another Python library that does?

Answer 1

Here are some links related to paging in python-ldap.

• Documentation: http://www.python-ldap.org/doc/html/ldap-controls.html#ldap.controls.SimplePagedResultsControl
• Example code using paging: http://www.novell.com/coolsolutions/tip/18274.html
• More example code: http://google-apps-for-your-domain-ldap-sync.googlecode.com/svn/trunk/ldap_ctxt.py

Answer 2

After some discussion on the python-ldap-dev mailing list, I can answer my own question.

Page controls ARE supported by the Python lDAP module, but the docs had not been updated for search_ext to show that. The example linked by Gorgapor shows how to use the ldap.controls.SimplePagedResultsControl to read the results in pages.

However there is a gotcha. This will work with Microsoft Active Directory servers, but not with OpenLDAP servers (and possibly others, such as Sun's). The LDAP controls RFC is ambiguous as to whether paged controls should be allowed to override the server's sizelimit setting. On ActiveDirectory servers they can by default while on OpenLDAP they cannot, but I think there is a server setting that will allow them to.

So even if you implement the paged control, there is still no guarantee that it will get all the objects that you want. Sigh

Also paged controls are only available with LDAP v3, but I doubt that there are many v2 servers in use.