How to join unknown number of lists in LINQ

Question

I have three lists of different types :

List<Customer> customerList = new List<Customer>();
List<Product> productList = new List<Product>();
List<Vehicle> vehicleList = new List<Vehicle>();

I also have this list

List<string> stringList = {"AND","OR"};

Since first element of stringList is AND I want to make inner join with customerList and productList. Then I want to make right join vehicleList with the result such as :

from cust in customerList 
join prod in productList on cust.ProductId equals prod.Id
join veh in vehicleList on prod.VehicleId equals veh.Id into v
from veh in v.DefaultIfEmpty()
select new {customerName = cust.Name, customerVehicle=veh.VehicleName}

I want to make this in automatized way, lets say I have N number of lists and N-1 number of ANDs and ORs, how can I join them? Besides there can be many lists of the same type. Is such a thing even possible? If not what can I do to make this closer to my need? Thanks in advance.

EDIT : I'm holding the lists and their types in a Dictionary like this :

var listDict = new Dictionary<Type, object>();

So I can iterate inside this dictionary if necessary.

Answer 1

UPDATE 5-15-17:

Just for the sake of recap what I am proposing is an example that we want to:

1. Pass in a list of N number of Table objects.
2. Pass in a list of N-1 join clauses of how to join them. EG: You have 2 tables you need a single join, 3 you need 2, and so on.
3. We want to be to pass in a predicate to go up or down the chain to narrow scope.

What I would propose is to do all of this in SQL and pass into SQL an xml object that it can parse. However to keep it a little more simple to not deal with XML serialization too, let's stick with strings that are essentially one or many values to pass in. Say we have a structure going off of above like this:

/*
CREATE TABLE Customer ( Id INT IDENTITY, CustomerName VARCHAR(64), ProductId INT)
INSERT INTO Customer VALUES ('Acme', 1),('Widgets', 2)
CREATE TABLE Product (Id INT IDENTITY, ProductName VARCHAR(64), VehicleId INT)
Insert Into Product Values ('Shirt', 1),('Pants', 2)
CREATE TABLE VEHICLE (Id INT IDENTITY, VehicleName VARCHAR(64))
INSERT INTO dbo.VEHICLE VALUES ('Car'),('Truck')

CREATE TABLE Joins (Id INT IDENTITY, OriginTable VARCHAR(32), DestinationTable VARCHAR(32), JoinClause VARCHAR(32))
INSERT INTO Joins VALUES ('Customer', 'Product', 'ProductId = Id'),('Product', 'Vehicle', 'VehicleId = Id')

--Data as is if I joined all three tables
CustomerId  CustomerName    ProductId   ProductName VehicleId   VehicleName
1   Acme    1   Shirt   1   Car
2   Widgets 2   Pants   2   Truck
*/

This structure is pretty simplistic and everything is one to one key relationships versus it could have some other identifiers. The key to making things work is to maintain a table that describes HOW these tables relate. I called this table joins. Now I can create a dynamic proc like so:

CREATE PROC pDynamicFind
  (
    @Tables varchar(256)
  , @Joins VARCHAR(256)
  , @Predicate VARCHAR(256)
  )
AS
BEGIN
  SET NOCOUNT ON;

    DECLARE @SQL NVARCHAR(MAX) = 
'With x as 
    (
    SELECT
    a.Id
  , {nameColumns}
  From {joins}
  Where {predicate}
  )
SELECT *
From x
  UNPIVOT (Value FOR TableName In ({nameColumns})) AS unpt
'
    DECLARE @Tbls TABLE (id INT IDENTITY, tableName VARCHAR(256), joinType VARCHAR(16))
    DECLARE @Start INT = 2
    DECLARE @alphas VARCHAR(26) = 'abcdefghijklmnopqrstuvwxyz'

    --Comma seperated into temp table (realistically most people create a function to do this so you don't have to do it over and over again)
    WHILE LEN(@Tables) > 0
    BEGIN
        IF PATINDEX('%,%', @Tables) > 0
        BEGIN
            INSERT INTO @Tbls (tableName) VALUES (RTRIM(LTRIM(SUBSTRING(@Tables, 0, PATINDEX('%,%', @Tables)))))
            SET @Tables = SUBSTRING(@Tables, LEN(SUBSTRING(@Tables, 0, PATINDEX('%,%', @Tables)) + ',') + 1, LEN(@Tables))
        END
        ELSE
        BEGIN
            INSERT INTO @Tbls (tableName) VALUES (RTRIM(LTRIM(@Tables)))
            SET @Tables = NULL
        END
    END

    --Have to iterate over this one seperately
    WHILE LEN(@Joins) > 0
    BEGIN
        IF PATINDEX('%,%', @Joins) > 0
        BEGIN
            Update @Tbls SET joinType = (RTRIM(LTRIM(SUBSTRING(@Joins, 0, PATINDEX('%,%', @Joins))))) WHERE id = @Start
            SET @Joins = SUBSTRING(@Joins, LEN(SUBSTRING(@Joins, 0, PATINDEX('%,%', @Joins)) + ',') + 1, LEN(@Joins))
            SET @Start = @Start + 1
        END
        ELSE
        BEGIN
            Update @Tbls SET joinType = (RTRIM(LTRIM(@Joins))) WHERE id = @Start
            SET @Joins = NULL
            SET @Start = @Start + 1
        END
    END

    DECLARE @Join VARCHAR(256) = ''
    DECLARE @Cols VARCHAR(256) = ''

    --Determine dynamic columns and joins
    Select 
      @Join += CASE WHEN joinType IS NULL THEN t.tableName + ' ' + SUBSTRING(@alphas, t.id, 1) 
      ELSE ' ' + joinType + ' JOIN ' + t.tableName + ' ' + SUBSTRING(@alphas, t.id, 1) + ' ON ' + SUBSTRING(@alphas, t.id-1, 1) + '.' + REPLACE(j.JoinClause, '= ', '= ' + SUBSTRING(@alphas, t.id, 1) + '.' )
      END
    , @Cols += CASE WHEN joinType IS NULL THEN t.tableName + 'Name' ELSE ' , ' + t.tableName + 'Name' END
    From @Tbls t
      LEFT JOIN Joins j ON t.tableName = j.DestinationTable

    SET @SQL = REPLACE(@SQL, '{joins}', @Join)
    SET @SQL = REPLACE(@SQL, '{nameColumns}', @Cols)
    SET @SQL = REPLACE(@SQL, '{predicate}', @Predicate)

    --PRINT @SQL
    EXEC sp_executesql @SQL
END
GO

I now have a medium for finding things that makes it stubbed query so to speak that I can replace the source of the from statement, what I query on, what value I use to query on. I would get results from it like this:

EXEC pDynamicFind 'Customer, Product', 'Inner', 'CustomerName = ''Acme'''
EXEC pDynamicFind 'Customer, Product, Vehicle', 'Inner, Inner', 'VehicleName = ''Car'''

Now what about setting that up in EF and using it in code? Well you can add procs to EF and get data from this as context. The answer that this addresses is that I am essentially giving back a fixed object now despite however many columns I may add. If my pattern is always going to be '(table)name' to N numbers of tables I can normalize my result by unpivoting and then just getting N number of rows for however many tables I have. Thus performance may be worse as you get larger result sets but the potential to make however many joins you want as long as similar structure is used is possible.

The point I am making though is that SQL is ultimately getting your data and doing crazy joins that result from Linq is at times more work than it's worth. But if you do have a small result set and a small db, you are probably fine. This is just an example of how you would get completely different objects in SQL using dynamic sql and how fast it can do something once the code for the proc is written. This is just one way to skin a cat of which I am sure there are many. The problem is whatever road you go down with dynamic joins or a method of getting things out is going to require some type of normalization standard, factory pattern or something where it says I can have N inputs that always yield the same X object no matter what. I do this through a vertical result set, but if you want a different column than say 'name' you are going to have to code more for that as well. However the way I built this if you want the description but say wanted to do a predicate for a date field, this would be fine with that.

Answer 2

If you always want the same set of output columns, then write your query ahead of time:

select * 
from

  customerList c
  inner join 
  productList p on c.ProductId = p.Id

  inner join
  vehicleList v on p.VehicleId = v.Id

Then append a dynamic where. At its simplest, just replace 'CustomerCity:' with 'c.city' and so on, so that what they wrote becomes valid SQL (Danger danger: if your user is not to be trusted then you must must must make your SQL injection proof. At the very least scan it for DML, or limit the keywords they can provide. Better would be to parse it into fields, parameterise it properly and add the values they provide to parameters)

Simple (ugh) we let the SQL parser do some work:

string whereClause = userInput;
whereClause = whereClause.Replace("CustomerCity:", "c.City = '");
whereClause = whereClause.Replace("VehicleNumber:", "v.Number = ");
//and so on
whereClause = whereClause.Replace(" AND", "' AND");
//some logic here to go through the string and close up those apostrophes

Ugly, and fragile. And hackable (if you care).

Parsing would be better:

sqlCommand.CommandText = "SELECT ... WHERE ";

string whereBits = userInput.Split(" ");
var parameters as new Dictionary<string, string>();
parameters["customercity"] = "c.City";
parameters["vehiclenumber"] = "v.Number";

foreach(var token in whereBits){
    var frags = token.Split(':');
    string friendlyName = frags[0].ToLower();

    //handle here the AND and OR -> append to sql command text and continue the loop        

    if(parameters.ContainsKey(friendlyName)){
      sqlCommand.CommandText += parameters[friendlyName] + " = @" + friendlyName;
      sqlCommand.Parameters.AddWithValue("@" + friendlyname, frags[1]);
    }
}

//now you should have an sql that looks like
//SELECT ... WHERE customercity = @customercity ...
// and a params collection that looks like:
//sql.Params[0] => ("@customercity", "Seattle", varchar)...

One thing to consider: will your user be able to construct that query and get the results they want? What in a users mind does CustomerCity:Seattle OR ProductType:Computer AND VehicleNumber:8 AND CustomerName:Jason mean anyway? Everyone in Seattle, plus every Jason whose Computer is in vehicle 8? Everyone in Seattle or who has a computer, but they must have vehicle 8 and be called jason?

Without precedence, queries could just turn out garbage in the user's hands