How to fetch more than 5000 entities from CRM

Question

I'm querying MS Dynamics CRM Online from my console app:

public EntityCollection GetEntities(string entityName)
{
    IOrganizationService proxy = ServerConnection.GetOrganizationProxy();

    string request = string.Format("<fetch mapping ='logical'><entity name = '{0}'></entity></fetch>", entityName);
    FetchExpression expression = new FetchExpression(request);
    var mult = proxy.RetrieveMultiple(expression);

    return mult;
}

This code only returns maximum of 5000 elements in mult.Entities. I know there are more entities in CRM. How to retrieve all entites?

Answer 1

You can use LINQ as shown below. The CRM LINQ provider will automatically page the query and run as many requests as is needed to get the full result, and return the complete set in a single object. It's really convenient for us. However, be careful with that. If the result set is very large it will be noticeably slow, and it could even throw an OutOfMemoryException in extreme cases.

 public List<Entity> GetEntities(string entityName)
    {
        OrganizationServiceContext DataContext = new OrganizationServiceContext(ServerConnection.GetOrganizationProxy());

        return DataContext.CreateQuery(entityName).toList();
    }

Here's an alternative implementation for paging with FetchXML, which I like much better than the official examples:

int page = 1;
EntityCollection entityList = new EntityCollection();

do
{
    entityList = Context.RetrieveMultiple(new FetchExpression(String.Format("<fetch version='1.0' page='{1}' paging-cookie='{0}' count='1000' output-format='xml-platform' mapping='logical' distinct='false'> ... </fetch>", SecurityElement.Escape(entityList.PagingCookie), page++)));

    // Do something with the results here
}
while (entityList.MoreRecords);