Merging two datatable in memory and grouping them to get sum of columns.Using linq but kind of lost here

Question

I have two table where two column are fixed. Some columns are identical and some are new.Columns are dynamic.

Have to do it in code level and I am trying to loop and conditions

What I want is to generate a report following the condition,

• All columns in table1 and table2 must be present.
• If a column is common and value is there it should be added with the identical row in other table.
• If any row is present in one table but not in other, it should be included.

Example data

Table1

ID  | NAME | P1  | P2  | P3 
 ----------------------------
  1  | A1   | 1   |  2 | 3.3
  2  | A2   | 4.4 |  5 |  6

TABLE 2

  ID  | NAME | P1 | P2  | P4 
 ---------------------------
  1  | A1   | 10  | 11  |  12
  2  | A2   | 12  | 14  |  15
  3  | A3   | 16  | 17  |  18

Expected output:

 ID  | NAME | P1   |  P2  | P3  | P4
 ---------------------------------
  1  | A1   | 11   |  13  | 3.3 | 12
  2  | A2   | 16.4 |  19  |  6  | 15
  3  | A3   | 16   |  17  | null| 18

Progress till now:

First I merged those two table in to table1 table1.Merge(table2)

Then trying to group by over it

var query = from row in table1.AsEnumerable()
                        group row by new
                        {
                            ID = row.Field<int>("ID"),
                            Name = row.Field<string>("Name")
                        }
                            into grp
                            select new
                            {
                                ID = grp.Key.ID,
                                Name = grp.Key.Name,
                                Phase1 = grp.Sum(r => r.Field<decimal>("P1"))
                            };

I have modified this code to get a datatable. Please see attached cs file.
This is working, but as the number of columns are dynamic, I guess I have to repeat it for other columns and join all these small tables where one columns will be added.

How can I merge all those small tables?

I am lost here.Is there any other way. Its feeling as stupid thing.

Any help would be appreciated.

Attached File:

---

http://dl.dropbox.com/u/26252340/Program.cs

Answer 1

You want to use an implementation of a full outer join. Something like what follows.

Some setup so you can try this yourself:

DataTable t1 = new DataTable();
t1.Columns.Add("ID", typeof(int));
t1.Columns.Add("Name", typeof(string));
t1.Columns.Add("P1", typeof(double));
t1.Columns.Add("P2", typeof(double));
t1.Columns.Add("P3", typeof(double));

DataRow dr1 = t1.NewRow();
dr1["ID"] = 1;
dr1["Name"] = "A1";
dr1["P1"] = 1;
dr1["P2"] = 2;
dr1["P3"] = 3.3;
t1.Rows.Add(dr1);
DataRow dr2 = t1.NewRow();
dr2["ID"] = 2;
dr2["Name"] = "A2";
dr2["P1"] = 4.4;
dr2["P2"] = 5;
dr2["P3"] = 6;
t1.Rows.Add(dr2);

DataTable t2 = new DataTable();
t2.Columns.Add("ID", typeof(int));
t2.Columns.Add("Name", typeof(string));
t2.Columns.Add("P1", typeof(double));
t2.Columns.Add("P2", typeof(double));
t2.Columns.Add("P4", typeof(double));

DataRow dr3 = t2.NewRow();
dr3["ID"] = 1;
dr3["Name"] = "A1";
dr3["P1"] = 10;
dr3["P2"] = 11;
dr3["P4"] = 12;
t2.Rows.Add(dr3);
DataRow dr4 = t2.NewRow();
dr4["ID"] = 2;
dr4["Name"] = "A2";
dr4["P1"] = 12;
dr4["P2"] = 14;
dr4["P4"] = 15;
t2.Rows.Add(dr4);
DataRow dr5 = t2.NewRow();
dr5["ID"] = 3;
dr5["Name"] = "A3";
dr5["P1"] = 16;
dr5["P2"] = 17;
dr5["P4"] = 18;
t2.Rows.Add(dr5);

The queries look like:

var ids = (from r1 in t1.AsEnumerable() select new { ID = r1["ID"], Name = r1["Name"] }).Union(
                    from r2 in t2.AsEnumerable() select new { ID = r2["ID"], Name = r2["Name"] });
var query = from id in ids
            join r1 in t1.AsEnumerable() on id equals new { ID = r1["ID"], Name = r1["Name"] } into left
            from r1 in left.DefaultIfEmpty()
            join r2 in t2.AsEnumerable() on id equals new { ID = r2["ID"], Name = r2["Name"] } into right
            from r2 in right.DefaultIfEmpty()    
            select new
            {
                ID = (r1 == null) ? r2["ID"] : r1["ID"],
                Name = (r1 == null) ? r2["Name"] : r1["Name"],
                P1 = (r1 == null) ? r2["P1"] : (r2["P1"] == null) ? r1["P1"] : (double)r1["P1"] + (double)r2["P1"],
                P2 = (r1 == null) ? r2["P2"] : (r2["P2"] == null) ? r1["P2"] : (double)r1["P2"] + (double)r2["P2"],
                P3 = (r1 == null) ? null : r1["P3"],
                P4 = (r2 == null) ? null : r2["P4"]
            };

Answer 2

Got this solved by

table1.Merge(table2, true, MissingSchemaAction.Add);
finalTable = table1.Clone();
finalTable.PrimaryKey = new DataColumn[] { finalTable.Columns["ID"], finalTable.Columns["Name"] };
List<string> columnNames = new List<string>();
for (int colIndex = 2; colIndex < finalTable.Columns.Count; colIndex++)
{
columnNames.Add(finalTable.Columns[colIndex].ColumnName);
}
foreach (string cols in columnNames)
{
var temTable = new DataTable();
temTable.Columns.Add("ID", typeof(int));
temTable.Columns.Add("Name", typeof(string));
temTable.Columns.Add(cols, typeof(decimal));

(from row in table1.AsEnumerable()
group row by new { ID = row.Field<int>("ID"), Team = row.Field<string>("Team") } into grp
orderby grp.Key.ID
select new
{
ID = grp.Key.ID,
Name = grp.Key.Team,
cols = grp.Sum(r =>  r.Field<decimal?>(cols)),
})
.Aggregate(temTable, (dt, r) => { dt.Rows.Add(r.ID, r.Team, r.cols); return dt; });

finalTable.Merge(temTable, false, MissingSchemaAction.Ignore);
}