Build one datatable out of two with certain conditions

Question

Firstly I need to get all the data from ODBC (this is working already).

Then comes the most complicated part that I am not sure yet how it can be done. There are two tables of data in ODBC. I am merging them with my current code and filtering them with certain parameters.

Table 1 in database:

NRO   NAME   NAMEA   NAMEB   ADDRESS   POSTA   POSTN   POSTADR   COMPANYN   COUNTRY   ID  ACTIVE
123   Fiat   Punto   500     J5        K4      O3      P4        O2         JT        1   1
133   Opel   Meriva  FTG     J5        K4      O3      P4        O2         JO        3   1
153   MB     E200    C25     JN        KI      OP      PY        OR         JD        5   1
183   BMW    E64     SE0     JR        KE      OT      PG        OL         J8        9   1
103   Audi   S6      700     JP        KU      OU      PN        OH         J6        11  1 

Table 2 in database:

NRO   NAME   NAMEA   NAMEB   ADDRESS   POSTA   POSTN   POSTADR   COMPANYN   COUNTRY   ID  ACTIVE
423   Fiat   Punto   500     J5        K4      O3      P4        O2         JT        1   1
463   BMW    E64     SE0     JR        KE      OT      PG        OL         J8        9   1

Merged dataTable look like this:

NRO   NAME   NAMEA   NAMEB   ADDRESS   POSTA   POSTN   POSTADR   COMPANYN   COUNTRY   ID  ACTIVE
423   Fiat   Punto   500     J5        K4      O3      P4        O2         JT        1   1
463   BMW    E64     SE0     JR        KE      OT      PG        OL         J8        9   1
123   Fiat   Punto   500     J5        K4      O3      P4        O2         JT        1   1
133   Opel   Meriva  FTG     J5        K4      O3      P4        O2         JO        3   1
153   MB     E200    C25     JN        KI      OP      PY        OR         JD        5   1
183   BMW    E64     SE0     JR        KE      OT      PG        OL         J8        9   1
103   Audi   S6      700     JP        KU      OU      PN        OH         J6        11  1 

However merged output dataTable should look like this (to have a possibility to work with it further):

NRO  NRO1   NAME   NAMEA   NAMEB   ADDRESS   POSTA   POSTN   POSTADR   COMPANYN   COUNTRY   ID  ACTIVE
123  423    Fiat   Punto   500     J5        K4      O3      P4        O2         JT        1   1
133         Opel   Meriva  FTG     J5        K4      O3      P4        O2         JO        3   1
153         MB     E200    C25     JN        KI      OP      PY        OR         JD        5   1
183  463    BMW    E64     SE0     JR        KE      OT      PG        OL         J8        9   1
103         Audi   S6      700     JP        KU      OU      PN        OH         J6        11  1 

Find duplicates in NAME. Leave only one of them, assign a number from Table 1 to NRO from Table 2 to NRO1. Table 1 numbers should be in NRO, Table 2 numbers should be in NRO1.

After connecting to ODBC I am filling one table with data from Table 1

        DataTable dataTable = new DataTable("COMPANY");

        using (OdbcConnection dbConnectionSE = new OdbcConnection(connectionStringSE))
        {
            dbConnectionSE.Open();
            OdbcDataAdapter dadapterSE = new OdbcDataAdapter();
            dadapterSE.SelectCommand = new OdbcCommand(queryStringSE, dbConnectionSE);

            dadapterSE.Fill(dataTable);

        }

then I am getting data from another Table 2 and merging them by:

         using (OdbcConnection dbConnectionFI = new OdbcConnection(connectionStringFI))
         {
              dbConnectionFI.Open();
              OdbcDataAdapter dadapterFI = new OdbcDataAdapter();
              dadapterFI.SelectCommand = new OdbcCommand(queryStringFI, dbConnectionFI);

              var newTable = new DataTable("COMPANY");
              dadapterFI.Fill(newTable);

              dataTable.Merge(newTable);
          }

After that I am performing filtering (I need to have rows only starting with 4 and 1 in NRO, there are also rows with other starting number):

DataTable results = dataTable.Select("ACTIVE = '1' AND (NRO Like '1%' OR NRO Like '4%')").CopyToDataTable();

Then I am adding one more Column for NRO1 (this is also adding zeros (0) I don't need them in Column NRO1):

        results.Columns.Add("NRO1", typeof(int)).SetOrdinal(1);

        foreach (DataRow row in results.Rows)
        {
            //need to set value to NewColumn column
            row["NRO1"] = 0;   // or set it to some other value
        }

I can catch duplicates with this code

var duplicates = results.AsEnumerable().GroupBy(r => r[2]).Where(gr => gr.Count() > 1);

but how to perform the rest? This should be performed by a loop with building a new table? How I can perform joining and removing duplicates to dataTable?

Answer 1

Try this:

1. Include NRO1 field in both queries for Table1 and Table2

2. Set default value 0 of NRO1 for Table1 (modify queryStringSE)

   e.g.:SELECT NRO,0 AS NRO1, NAME,NAMEA,NAMEB, ... FROM TABLE1

3. Set default value 0 of NRO for Table2 (modify queryStringFI)

   e.g.:SELECT 0 AS NRO,NRO AS NRO1,NAME,NAMEA,NAMEB,...... FROM TABLE2

Table1 will look like:

NRO  NRO1   NAME   NAMEA   NAMEB   ADDRESS   POSTA   POSTN   POSTADR   COMPANYN   COUNTRY   ID  ACTIVE
123   0     Fiat   Punto   500     J5        K4      O3      P4        O2         JT        1   1
133   0     Opel   Meriva  FTG     J5        K4      O3      P4        O2         JO        3   1

Table2 will look like:

NRO  NRO1   NAME   NAMEA   NAMEB   ADDRESS   POSTA   POSTN   POSTADR   COMPANYN   COUNTRY   ID  ACTIVE
0    423    Fiat   Punto   500     J5        K4      O3      P4        O2         JT        1   1
0    463    BMW    E64     SE0     JR        KE      OT      PG        OL         J8        9   1

4. Merge tables as you are already doing

Add following lines of code:

var carGroups = dataTable.AsEnumerable().GroupBy(row => new 
{
   Name = row.Field<string>("Name"),
   NameA = row.Field<string>("NAMEA"),
   NameB = row.Field<string>("NAMEB")
   //Other fields.....
});

DataTable result = dataTable.Clone();

foreach(var grp in carGroups)            
    result.Rows.Add(grp.Sum(r1 => r1.Field<int>("NRO")), grp.Sum(r2 => r2.Field<int>("NRO1")), grp.Key.Name, grp.Key.NameA, grp.Key.NameB);              

5. Check DataTable "result" for desired values

Answer 2

You could replace the merge() call with a custom method, which does the merging and filtering at the same time. See the example below. I think this is a better approach than first merging (introducing duplicate rows in the result table) and then filtering (i.e. removing the duplicate rows).

Here, it is assumed that the parameters all have the same format. The tTemp table is used as a temporary storage for the contents of table t2 but with the extra column. This allows importing the rows in the result table.

Maybe there is a more elegant solution, but this should work as intended. Please note that I have left out your additional requirement regarding the allowed values for NRO, which I am sure you can add easily.

static void merge_it(DataTable t1, DataTable t2, DataTable tResult, DataTable tTemp)
    {
        tResult.Merge(t1);
        tResult.Columns.Add("NRO1", typeof(int));

        tTemp.Merge(t2);
        tTemp.Columns.Add("NRO1", typeof(int));

        foreach (DataRow row in tTemp.Rows)
        {
            string name1 = row.Field<string>("NAME");
            string name2 = row.Field<string>("NAMEA");
            DataRow[] matches = tResult.Select($"NAME = '{name1}' AND NAMEA = '{name2}'");
            if (matches.Length > 0)
            {
                matches[0].SetField<int>("NRO1", row.Field<int>("NRO"));
            }
            else
            {
                tResult.ImportRow(row);
            }
        }

        foreach (DataRow row in tResult.Rows)
        {
            if (row["NRO1"] == DBNull.Value)
            {
                row["NRO1"] = 0;
            }
        }
    }

Answer 3

you can keep same column name in both table if they denote same type of entity then see this code

 private static void DemonstrateMergeTable()
{
    DataTable table1 = new DataTable("Items");

    // Add columns
    DataColumn idColumn = new DataColumn("id", typeof(System.Int32));
    DataColumn itemColumn = new DataColumn("item", typeof(System.Int32));
    table1.Columns.Add(idColumn);
    table1.Columns.Add(itemColumn);

    // Set the primary key column.
    table1.PrimaryKey = new DataColumn[] { idColumn };

    // Add RowChanged event handler for the table.
    table1.RowChanged += new 
        System.Data.DataRowChangeEventHandler(Row_Changed);

    // Add ten rows.
    DataRow row;
    for (int i = 0; i <= 9; i++)
    {
        row = table1.NewRow();
        row["id"] = i;
        row["item"] = i;
        table1.Rows.Add(row);
    }

    // Accept changes.
    table1.AcceptChanges();
    PrintValues(table1, "Original values");

    // Create a second DataTable identical to the first.
    DataTable table2 = table1.Clone();

    // Add column to the second column, so that the 
    // schemas no longer match.
    table2.Columns.Add("newColumn", typeof(System.String));

    // Add three rows. Note that the id column can't be the 
    // same as existing rows in the original table.
    row = table2.NewRow();
    row["id"] = 14;
    row["item"] = 774;
    row["newColumn"] = "new column 1";
    table2.Rows.Add(row);

    row = table2.NewRow();
    row["id"] = 12;
    row["item"] = 555;
    row["newColumn"] = "new column 2";
    table2.Rows.Add(row);

    row = table2.NewRow();
    row["id"] = 13;
    row["item"] = 665;
    row["newColumn"] = "new column 3";
    table2.Rows.Add(row);

    // Merge table2 into the table1.
    Console.WriteLine("Merging");
    table1.Merge(table2, false, MissingSchemaAction.Add);
    PrintValues(table1, "Merged With table1, schema added");
}

private static void Row_Changed(object sender, 
    DataRowChangeEventArgs e)
{
    Console.WriteLine("Row changed {0}\t{1}", e.Action, 
        e.Row.ItemArray[0]);
}

private static void PrintValues(DataTable table, string label)
{
    // Display the values in the supplied DataTable:
    Console.WriteLine(label);
    foreach (DataRow row in table.Rows)
    {
        foreach (DataColumn col in table.Columns)
        {
            Console.Write("\t " + row[col].ToString());
        }
        Console.WriteLine();
    }
}