Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Bulk insert into partitioned table and table level lock

I want to know the core reason(the mechanics of segments, blocks, locks that the engine does) why bulk insert(with direct-path) locks the entire table so if I insert into a partition, I can't truncate another partition which is not affected(apparently) by insert.

A conventional insert(without append hint) permits to truncate some nonaffected partitions.(Notice that i speak about non-commited transaction.)

Below an example to ilustrate it.

Let be a table:

 CREATE TABLE FG_TEST 
   (COL NUMBER ) 
  PARTITION BY RANGE (COL) 
 (PARTITION "P1"  VALUES LESS THAN (1000), 
  PARTITION "P2"  VALUES LESS THAN (2000));

Insert into table fg_test values (1);
insert into table fg_test values (1000);
commit;

Session 1:

insert into table fg_test select * from fg_test where col >=1000;
--1 rows inserted;

Session 2:

alter table fg_test truncate partition p1;
--table truncated

Session 1:

rollback;
insert /*+append */ into table fg_test select * from fg_test where col >=1000;
--1 rows inserted;

Session 2:

alter table fg_test truncate partition p1;
--this throws ORA-00054: resource busy and acquire with NOWAIT specified 
--or timeout expired

The Doc on Diret-Path Insert is pretty abrupt on this subject and just says:

During direct-path INSERT, the database obtains exclusive locks on the table (or on all partitions of a partitioned table). As a result, users cannot perform any concurrent insert, update, or delete operations on the table, and concurrent index creation and build operations are not permitted.

The How Direct-Path INSERT Works does not explain why the lock is needed for all partitions. And why conventional insert does not lock nonaffected partitions? (My intuition is that the lock is done at block level)

like image 312
Florin stands with Ukraine Avatar asked May 10 '13 07:05

Florin stands with Ukraine


People also ask

What is Bulk insert table lock?

Specifies that a table-level lock is acquired for the duration of the bulk-import operation. A table can be loaded concurrently by multiple clients if the table has no indexes and TABLOCK is specified. By default, locking behavior is determined by the table option table lock on bulk load.

How do I lock a table while inserting in SQL Server?

To make it so that nobody can read from the table while you're inserting: insert into LargeTable with (tablockx) ... You don't have to do anything to make the table look empty until after the insert completes.

How can I speed up bulk insert in SQL Server?

Below are some good ways to improve BULK INSERT operations : Using TABLOCK as query hint. Dropping Indexes during Bulk Load operation and then once it is completed then recreating them. Changing the Recovery model of database to be BULK_LOGGED during the load operation.

How do I insert data into a specific partition in Oracle?

You can use the INSERT statement to insert data into a table, partition, or view in two ways: conventional INSERT and direct-path INSERT . When you issue a conventional INSERT statement, Oracle Database reuses free space in the table into which you are inserting and maintains referential integrity constraints.

What happens when data is bulk imported to the partitioned table(s)?

When data is bulk imported to the partitioned table (s), records are distributed among the filegroups according to a partition scheme, as described below. 1. Create a partition function

How do I partition a table in SQL Server?

Create a partition table Create partitioned table (s) according to your data schema, and specify the partition scheme and constraint field used to partition the table, for example: For more information, see Create Partitioned Tables and Indexes. You may use BCP, BULK INSERT, or other methods such as Microsoft Data Migration.

What is tablock in SQL Server?

Specifies that a table-level lock is acquired for the duration of the bulk-import operation. A table can be loaded concurrently by multiple clients if the table has no indexes and TABLOCK is specified. By default, locking behavior is determined by the table option table lock on bulk load.

How do you use a Bulk insert in a transaction?

The BULK INSERT statement can be executed within a user-defined transaction to import data into a table or view. Optionally, to use multiple matches for bulk importing data, a transaction can specify the BATCHSIZE clause in the BULK INSERT statement.


2 Answers

Your premise is slightly wrong. A direct-path insert does not lock the entire table if you use the partition extension clause.

Session 1:

insert /*+append */ into fg_test partition (p2)
select * from fg_test where col >=1000;

Session 2:

alter table fg_test truncate partition p1;
--table truncated

The new question is: When the partition extension clause is NOT used, why do conventional and direct-path inserts have different locking mechanisms? This clarification makes the question easier, but without inside knowledge the answer below is still only a guess.


It was easier to code a feature that locks the entire table. And it runs faster, since there is no need to track which partitions are updated.

There's usually no need for a more fine-grained lock. Most systems or processes that use direct-path writes only update one large table at a time. If a more fine-grained lock is really needed, the partition extension clause can be used. It's not quite as convenient, since only one partition can be referenced at a time. But it's good enough 99.9% of the time.

like image 130
Jon Heller Avatar answered Sep 20 '22 21:09

Jon Heller


I found the follwing answer on asktom.oracle.com:

Ask Tom: Inserts with APPEND Hint

Tom explains many of the inner workings, but the reason why Oracle locks the whole table and not only affected partitions is still not clear.

Maybe it's just a design decision (e.g. not wanting the big bulky direct load to be potentially blocked by one smallish uncommited transaction and therefore locking all partitions ...)

like image 34
HAL 9000 Avatar answered Sep 20 '22 21:09

HAL 9000