Get a big List<T> from WCF in chunks?

Question

I'm trying to get a list of entities from a WCF service, the problem I'm experiencing is that we have some bad latency on the network and so the data takes a considerable amount of time to get to my client. The idea I have is to find a way to get the first 1000 and just push those to the UI while I'm waiting for the next ones to arrive.

I guess it would be like paging but I just want to page the full set in the WCF layer rather that get one page at a time from the db

Cheers

Answer 1

WCF looks at the message at its entirety before handing it to higher levels. Hence your data needs to arrive in full and usual WCF contracts will not work.

However, you can use streaming with WCF. This allows for the payload to be read gradually from the stream and be passed to the higher levels. In order to get it working you need to:

• enable streaming (in the link I provided)
• change service contract to provide a stream
• at server end, start serializing entities and writing to the stream, perhaps in 100 chunks. You need to serialize entities yourself although using DataContractSerializer our of the box is very simple.
• You need to provide a delimiter between each of these 100 entities so that your process could spot where current 100 is finished. One possible option is 1 KB of byte zero.
• at the client side, implement function as Async. Keep reading from the stream in buffers (e.g 4KB) until hit the delimiter. Once hit that, deserialize and raise async event.

This will more complex that WCF straight out of the box but achieves what you need.

Answer 2

You can always split your service interface into two methods. For example, instead of:

List<T> GetItems()

You can have:

int GetItemCount()

List<T> GetItems(int start, int maxSize)

So that you can implement paging manually.