In .Net 4.0, can DirectorySearch return LDAP results in a way that allows me to page through them?

Question

I am working in C#, and am trying to use DirectorySearch to query the groups of an extremely large Microsoft ActiveDirectory LDAP server.

So, in my application, I'm going to have a paged list of groups, with searching capability. Naturally, I don't want to hammer my LDAP server with passing me the entire result set for these queries every time I hit "Next Page".

Is there a way, using DirectorySearch, to retrieve ONLY a single arbitrary page's results, rather than returning the entire result-set in one method call?

Similar questions:

• DirectorySearch.PageSize = 2 doesn't work
• c# Active Directory Services findAll() returns only 1000 entries

Many questions like these exist, where someone asks about paging (meaning from LDAP server to app server), and gets responses involving PageSize and SizeLimit. However, those properties only affect paging between the C# server and the LDAP server, and in the end, the only relevant methods that DirectorySearch has are FindOne() and FindAll().

What I'm looking for is basically "FindPaged(pageSize, pageNumber)" (the pageNumber being the really important bit. I don't just want the first 1000 results, I want (for example) the 100'th set of 1000 results. The app can't wait for 100,000 records to be passed from the LDAP server to the app server, even if they are broken up into 1,000-record chunks.

I understand that DirectoryServices.Protocols has SearchRequest, which (I think?) allows you to use a "PageResultRequestControl", which looks like it has what I'm looking for (although it looks like the paging information comes in "cookies", which I'm not sure how I'd be supposed to retrieve). But if there's a way to do this without rewriting the entire thing to use Protocols instead, I'd rather not have to do so.

I just can't imagine there's no way to do this... Even SQL has Row_Number.

UPDATE: The PageResultRequestControl does not help - It's forward-only and sequential (You must call and get the first N results before you can get the "cookie" token necessary to make a call to get result N+1).

However, the cookie does appear to have some sort of reproducible ordering... On a result set I was working on, I iterated one by one through the results, and each time the cookie came out thusly:

1: {8, 0, 0, 0}
2: {11, 0, 0, 0}
3: {12, 0, 0, 0}
4: {16, 0, 0, 0}

When I iterated through two by two, I got the same numbers (11, 16). This makes me think that if I could figure out the code of how those numbers are generated, I could create a cookie ad-hoc, which would give me exactly the paging I'm looking for.

Answer 1

The PageResultRequestControl is indeed the way to do this, it's part of the LDAP protocol. You'll just have to figure out what that implies for your code, sorry. There should be a way to use it from where you are, but, having said that, I'm working in Java and I've just had to write a dozen or so request controls and extended-operation classes for use with JNDI so you might be out of luck ... or you might have to do like I did. Warning, ASN.1 parsing follows not that far behind :-|

Answer 2

Sadly, it appears there may not be a way to do this given current C# libraries.

All of the standard C#4.0 LDAP libraries return Top-N results (As in, FindAll(), which returns every result, FindOne(), which returns the first result, or SearchResult with PageResultRequestControl, which returns results N through N+M but requires you to retrieve results 1 through N-1 before you'll have a cookie token that you can pass with the request in order to get the next set.

I haven't been able to find any third-party LDAP libraries that allow this, either.

Unless a better solution is found, my path forward will be to modify the interface to instead display the top X results, with no client paging capabilities (obviously still using server-side paging as appropriate).

I may pursue a forward-only paging system at a later date, by passing the updated cookie to the client with the response, and passing it back with a click of a "More Results" type of button. It might be worth pursuing at a later date, whether or not these cookies can be hand-crafted.

UPDATE: I spoke with Microsoft Support and confirmed this - There is no way to do dynamic paging with LDAP servers. This is a limitation of LDAP servers themselves.

You can use Protocols and the Paging control (if your LDAP server supports it) to step forward at will, but there is no cross-server (or even cross-version) standard for the cookie, so you can't reasonably craft your own, and there's no guarantee that the cookie can be reused for repeated queries.

A full solution involves using Protocols (with Paging as above) to pull your pageable result set into SQL, whether into a temp table or a permanent storage table, and allow your user to page and sort through THAT result set in the traditional manner. Bear in mind your results won't be precisely up to date, but with some smart cache updating you can minimize that risk.