Why are batch inserts/updates faster? How do batch updates work?

Question

Why are batch inserts faster? Is it because the connection and setup overhead for inserting a single row is the same for a set of rows? What other factors make batch inserts faster?

How do batch updates work? Assuming the table has no uniqueness constraints, insert statements don't really have any effect on other insert statements in the batch. However, during batch updates, an update can alter the state of the table and hence can affect the outcome of other update queries in the batch.

I know that batch insert queries have a syntax where you have all the insert values in one big query. How do batch update queries look like? For e.g. if i have single update queries of the form:

update <table> set <column>=<expression> where <condition1> update <table> set <column>=<expression> where <condition2> update <table> set <column>=<expression> where <condition3> update <table> set <column>=<expression> where <condition4> 

What happens when they are used in a batch. What will the single query look like ?

And are batch inserts & updates part of the SQL standard?

Answer 1

I was looking for an answer on the same subject, about "bulk/batch" update. People often describe the problem by comparing it with insert clause with multiple value sets (the "bulk" part).

INSERT INTO mytable (mykey, mytext, myint) VALUES    (1, 'text1', 11),   (2, 'text2', 22),   ... 

Clear answer was still avoiding me, but I found the solution here: http://www.postgresql.org/docs/9.1/static/sql-values.html

To make it clear:

UPDATE mytable SET    mytext = myvalues.mytext,   myint = myvalues.myint FROM (   VALUES     (1, 'textA', 99),     (2, 'textB', 88),     ... ) AS myvalues (mykey, mytext, myint) WHERE mytable.mykey = myvalues.mykey 

It has the same property of being "bulk" aka containing alot of data with one statement.

Answer 2

Why are batch inserts faster?

For numerous reasons, but the major three are these:

• The query doesn't need to be reparsed.
• The values are transmitted in one round-trip to the server
• The commands are inside a single transaction

Is it because the connection and setup overhead for inserting a single row is the same for a set of rows?

Partially yes, see above.

How do batch updates work?

This depends on RDBMS.

In Oracle you can transmit all values as a collection and use this collection as a table in a JOIN.

In PostgreSQL and MySQL, you can use the following syntax:

INSERT INTO    mytable VALUES          (value1),         (value2),         … 

You can also prepare a query once and call it in some kind of a loop. Usually there are methods to do this in a client library.

Assuming the table has no uniqueness constraints, insert statements don't really have any effect on other insert statements in the batch. But, during batch updates, an update can alter the state of the table and hence can affect the outcome of other update queries in the batch.

Yes, and you may or may not benefit from this behavior.

I know that batch insert queries have a syntax where you have all the insert values in one big query. How do batch update queries look like?

In Oracle, you use collection in a join:

MERGE INTO    mytable USING   TABLE(:mycol) ON      … WHEN MATCHED THEN UPDATE SET     … 

In PostgreSQL:

UPDATE  mytable SET     s.s_start = 1 FROM    (         VALUES         (value1),         (value2),         …         ) q WHERE   …