Linq Should I Return List<T> Or IEnumerable<T> When I Still May Do More Later

Question

I have some methods that return a List of T like say GetAllEvents. In some cases I need to then filter that list of events (or whatever my List is) by date or some other property on the items.

I know that LINQ queries can be "chained" or have x number of lines that further refine them and the query will not execute until some point when you need to actually use them in a non-linq statement (please correct me if I am wrong on this beliefe.)

My question is, if my GetAllXXX method returns a List of whatever I am getting, is the .ToList() method I am using at the end of my GetAllXXX code executing the LINQ? Should I return IEnumerable instead? If only for those cases where I need to do something further to the "results" BEFORE the query is actually run.

Here is an example of my worry: I have say 1000 events. The GetAllEvents will retrieve all 1000 and give me a List of them. Then, depending on what page the user is on, may only show events for Today, this week or of a certain category. Ideally, by the time it gets to the point where I am showing the user the 5 events happening today, I don't really want to pass all 1000 across the wire and then truncate that down to the 5 they really want. Yes, I know it is all server side at this point, but if it is still allocating memory for the 1000 I am trying to avoid this.

Any pointers or suggestions?

Answer 1

If you turn the sequence into a list on the server then you are using time and memory on the server, and then transmitting the whole thing over the wire, and then using more time and memory on the client to filter the list.

If you just return the sequence then you are "solving" your problem by creating a different problem. Now when the client goes to filter the list they have to make a thousand small hits to the server instead of one expensive hit. Just as much information is going over the wire, and taking far longer with all the per-hit overhead.

If what you want to do is perform the filtering on the server then you can (1) create a custom API that represents common filters (easy), or (2) instead return IQueryable, and implement a LINQ provider. (hard, but powerful.) IQueryable lets you construct the query on the client side, send the query over the wire to the server, run the query on the server, and then only serve up the results the client wanted.

My colleague Matt Warren wrote a whole series of articles on how to implement IQueryable; I would start with that.

Answer 2

Return an IEnumerable.

The conversion to a List is quick and painless, plus you keep you interface decoupled from both the implementation of your methods and the use of the output.

Regarding your specific worry - if it is going to be expensive to return all 1000 events and process them on the client then you should consider doing some filtering on the server. You can still have a method that returns all the events, but have specialised/optimised versions that return the most frequent queries. The events for today would be a good example.

Answer 3

Eric Lippert's answer is good, but note that you do not need to implement the whole of IQueryable if you just want to provide some server-side filtering (even custom filtering). You can create a LINQ-compatible API more simpy by implementing only the LINQ functions that you'll actually use. For example, consider defining

interface IOneTripEnumerable<T> : IEnumerable<T>

which exposes only a LINQ-compatible Where method with return type IOneTripEnumerable

The implementation of IOneTripEnumerable.Where would return a new object that also implements IOneTripEnumerable and simply stores the filter as a data member. When IOneTripEnumerable.GetEnumerator is called you can package up the filters and send them to the server then get back the filtered results in one round-trip.

(You could also implement a client-side caching policy: if you want subsequent calls to GetEnumerator to return an enumerator over the same results as the initial call, just store the results in the enumerable object.)

If you get more time and see the need, you can optimize further by adding additional LINQ methods, but just taking control of Where (to allow the filtering to occur on the server) and GetEnumerator (to get all the results in a single round-trip) could give you pretty good results at low cost to implement. You don't need to implement the whole of IQueryable. (Note that Count, Any, and Take are also very good candidates for round-trip optimization and are trivial to implement).