Is there any way to optimize this LINQ to Entities query?

Question

I was asked to produce a report that is driven by a fairly complex SQL query against a SQL Server database. Since the site of the report was already using Entity Framework 4.1, I thought I would attempt to write the query using EF and LINQ:

var q = from r in ctx.Responses
                    .Where(x => ctx.Responses.Where(u => u.UserId == x.UserId).Count() >= VALID_RESPONSES)
                    .GroupBy(x => new { x.User.AwardCity, x.Category.Label, x.ResponseText })
         orderby r.FirstOrDefault().User.AwardCity, r.FirstOrDefault().Category.Label, r.Count() descending
         select new
         {
             City = r.FirstOrDefault().User.AwardCity,
             Category = r.FirstOrDefault().Category.Label,
             Response = r.FirstOrDefault().ResponseText,
             Votes = r.Count()
         };

This query tallies votes, but only from users who have submitted a certain number of required minimum votes.

This approach was a complete disaster from a performance perspective, so we switched to ADO.NET and the query ran very quickly. I did look at the LINQ generated SQL using the SQL Profiler, and although it looked atrocious as usual I didn't see any clues as to how to optimize the LINQ statement to make it more efficient.

Here's the straight TSQL version:

WITH ValidUsers(UserId)
AS
(
    SELECT UserId
    FROM Responses
    GROUP BY UserId
    HAVING COUNT(*) >= 103
)
SELECT d.AwardCity
    , c.Label
    , r.ResponseText
    , COUNT(*) AS Votes
FROM ValidUsers u
JOIN Responses r ON r.UserId = u.UserId
JOIN Categories c ON r.CategoryId = c.CategoryId
JOIN Demographics d ON r.UserId = d.Id
GROUP BY d.AwardCity, c.Label, r.ResponseText
ORDER BY d.AwardCity, s.SectionName, COUNT(*) DESC

What I'm wondering is: is this query just too complex for EF and LINQ to handle efficiently or have I missed a trick?

Answer 1

Using a let to reduce the number of r.First()'s will probably improve performance. It's probably not enough yet.

 var q = from r in ctx.Responses
                .Where()
                .GroupBy()
     let response = r.First()
     orderby response.User.AwardCity, response.Category.Label, r.Count() descending
     select new
     {
         City = response.User.AwardCity,
         Category = response.Category.Label,
         Response = response.ResponseText,
         Votes = r.Count()
     };