Bulk Update in C#

Question

What I've done before is perform a bulk insert from the data into a temp table, and then use a command or stored procedure to update the data relating the temp table with the destination table. The temp table is an extra step, but you can have a performance gain with the bulk insert and massive update if the amount of rows is big, compared to updating the data row by row.

Example:

public static void UpdateData<T>(List<T> list,string TableName)
{
    DataTable dt = new DataTable("MyTable");
    dt = ConvertToDataTable(list);

    using (SqlConnection conn = new SqlConnection(ConfigurationManager.ConnectionStrings["SchoolSoulDataEntitiesForReport"].ConnectionString))
    {
        using (SqlCommand command = new SqlCommand("", conn))
        {
            try
            {
                conn.Open();

                //Creating temp table on database
                command.CommandText = "CREATE TABLE #TmpTable(...)";
                command.ExecuteNonQuery();

                //Bulk insert into temp table
                using (SqlBulkCopy bulkcopy = new SqlBulkCopy(conn))
                {
                    bulkcopy.BulkCopyTimeout = 660;
                    bulkcopy.DestinationTableName = "#TmpTable";
                    bulkcopy.WriteToServer(dt);
                    bulkcopy.Close();
                }

                // Updating destination table, and dropping temp table
                command.CommandTimeout = 300;
                command.CommandText = "UPDATE T SET ... FROM " + TableName + " T INNER JOIN #TmpTable Temp ON ...; DROP TABLE #TmpTable;";
                command.ExecuteNonQuery();
            }
            catch (Exception ex)
            {
                // Handle exception properly
            }
            finally
            {
                conn.Close();
            }
        }
    }
}

Notice that a single connection is used to perform the whole operation, in order to be able to use the temp table in each step, because the scope of the temp table is per connection.

In my personal experience, the best way to handled this situation is utilizing a Stored Procedure with a Table-Valued Parameter and a User-Defined Table Type. Just set up the type with the columns of the data table, and pass in said-data table as a parameter in the SQL command.

Within the Stored Procedure, you can either join directly on some unique key (if all rows you are updating exist), or - if you might run into a situation where you are having to do both updates and inserts - use the SQL Merge command within the stored procedure to handle both the updates and inserts as applicable.

Microsoft has both syntax reference and an article with examples for the Merge.

For the .NET piece, it's a simple matter of setting the parameter type as SqlDbType.Structured and setting the value of said-parameter to the Data Table that contains the records you want to update.

This method provides the benefit of both clarity and ease of maintenance. While there may be ways that offer performance improvements (such as dropping it into a temporary table then iterating over that table), I think they're outweighed by the simplicity of letting .NET and SQL handle transferring the table and updating the records itself. K.I.S.S.

Bulk Update:

Step 1: put the data which you want to update and primary key in a list.

Step 2: pass this list and ConnectionString to BulkUpdate Method As shown below

Example:

         //Method for Bulk Update the Data
    public static void BulkUpdateData<T>(List<T> list, string connetionString)
    {

        DataTable dt = new DataTable("MyTable");
        dt = ConvertToDataTable(list);

        using (SqlConnection conn = new SqlConnection(connetionString))
        {
            using (SqlCommand command = new SqlCommand("CREATE TABLE 
                  #TmpTable([PrimaryKey],[ColumnToUpdate])", conn))
            {
                try
                {
                    conn.Open();
                    command.ExecuteNonQuery();

                    using (SqlBulkCopy bulkcopy = new SqlBulkCopy(conn))
                    {
                        bulkcopy.BulkCopyTimeout = 6600;
                        bulkcopy.DestinationTableName = "#TmpTable";
                        bulkcopy.WriteToServer(dt);
                        bulkcopy.Close();
                    }


                    command.CommandTimeout = 3000;
                    command.CommandText = "UPDATE P SET P.[ColumnToUpdate]= T.[ColumnToUpdate] FROM [TableName Where you want to update ] AS P INNER JOIN #TmpTable AS T ON P.[PrimaryKey] = T.[PrimaryKey] ;DROP TABLE #TmpTable;";
                    command.ExecuteNonQuery();
                }
                catch (Exception ex)
                {
                    // Handle exception properly
                }
                finally
                {
                    conn.Close();
                }
            }
        }
    }

Step 3: put The ConvertToDataTable Method as shown Below.

Example:

    public static DataTable ConvertToDataTable<T>(IList<T> data)
    {
        PropertyDescriptorCollection properties = TypeDescriptor.GetProperties(typeof(T));
        DataTable table = new DataTable();
        foreach (PropertyDescriptor prop in properties)
            table.Columns.Add(prop.Name, Nullable.GetUnderlyingType(prop.PropertyType) ?? prop.PropertyType);
        foreach (T item in data)
        {
            DataRow row = table.NewRow();
            foreach (PropertyDescriptor prop in properties)
                row[prop.Name] = prop.GetValue(item) ?? DBNull.Value;
            table.Rows.Add(row);
        }
        return table;
    }

Notes: WhereEver SquareBracket[] is there, put your own value.

Try out SqlBulkTools available on Nuget.

Disclaimer: I'm the author of this library.

var bulk = new BulkOperations();
var records = GetRecordsToUpdate();

using (TransactionScope trans = new TransactionScope())
{
    using (SqlConnection conn = new SqlConnection(ConfigurationManager
    .ConnectionStrings["SqlBulkToolsTest"].ConnectionString))
    {
        bulk.Setup<MyTable>()
            .ForCollection(records)
            .WithTable("MyTable")
            .AddColumn(x => x.SomeColumn1)
            .AddColumn(x => x.SomeColumn2)
            .BulkUpdate()
            .MatchTargetOn(x => x.Identifier)
            .Commit(conn);
    }

    trans.Complete();
}  

Only 'SomeColumn1' and 'SomeColumn2' will be updated. More examples can be found here

I would insert new values in a temporary table and then do a merge against the destination table, something like this:

MERGE [DestTable] AS D 
USING #SourceTable S
    ON D.ID = S.ID
WHEN MATCHED THEN 
    UPDATE SET ...
WHEN NOT MATCHED 
THEN INSERT (...) 
VALUES (...);

You could try to build a query that contains all data. Use a case. It could look like this

update your_table
set some_column = case when id = 1 then 'value of 1'
                       when id = 5 then 'value of 5'
                       when id = 7 then 'value of 7'
                       when id = 9 then 'value of 9'
                  end
where id in (1,5,7,9)

I'd go for a TempTable approach because that way you aren't locking anything. But if your logic needs to be only in the front end and you need to use bulk copy, I'd try a Delete/Insert approach but in the same SqlTransaction to ensure integrity which would be something like this:

// ...

dt = ConvertToDataTable(list);

using (SqlConnection cnx = new SqlConnection(myConnectionString))
{
    using (SqlTranscation tran = cnx.BeginTransaction())
    {
        DeleteData(cnx, tran, list);

        using (SqlBulkCopy bulkcopy = new SqlBulkCopy(cnx, SqlBulkCopyOptions.Default, tran))
        {
            bulkcopy.BulkCopyTimeout = 660;
            bulkcopy.DestinationTableName = TabelName;
            bulkcopy.WriteToServer(dt);
        }

        tran.Commit();
    }
}