Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to efficiently delete rows while NOT using Truncate Table in a 500,000+ rows table

Let's say we have table Sales with 30 columns and 500,000 rows. I would like to delete 400,000 in the table (those where "toDelete='1'").

But I have a few constraints :

  • the table is read / written "often" and I would not like a long "delete" to take a long time and lock the table for too long
  • I need to skip the transaction log (like with a TRUNCATE) but while doing a "DELETE ... WHERE..." (I need to put a condition), but haven't found any way to do this...

Any advice would be welcome to transform a

DELETE FROM Sales WHERE toDelete='1' 

to something more partitioned & possibly transaction log free.

like image 752
Skippy Fastol Avatar asked Jun 27 '12 15:06

Skippy Fastol


People also ask

Which is more efficient delete or truncate?

TRUNCATE is faster than DELETE , as it doesn't scan every record before removing it. TRUNCATE TABLE locks the whole table to remove data from a table; thus, this command also uses less transaction space than DELETE . Unlike DELETE , TRUNCATE does not return the number of rows deleted from the table.

How do you delete 10000 records in SQL?

If you need to remove 10 million rows and have 1 GB of log space available use Delete TOP(10000) From dbo. myTable (with your select clause) and keep running it till there are no more rows to delete.

How to delete all rows from a big table in SQL?

To delete all rows from a big table fast, you use the following TRUNCATE TABLE statement: In this syntax, you specify the table_name that you want to delete data after the TRUNCATE TABLE clause.

What is the difference between delete and truncate in SQL?

The DELETE statement with a WHERE clause deletes partial data from a table while the TRUNCATE TABLE statement always removes all data from the table. Let’s take a look at an example of truncating a table.

How can I remove large amounts of data from a table?

If you are willing (and able) to implement partitioning, that is an effective technique for removing large quantities of data with little run-time overhead. Not cost-effective for a once-off exercise, though. Show activity on this post. I was able to delete 19 million rows from my table of 21 million rows in matter of minutes. Here is my approach.

Is it faster to delete or insert rows in a table?

Inserting rows in a table is faster than deleting them. Loading data into a new table using create-table-as-select (CTAS) is faster still. So if you're removing most of the rows from a table, instead of issuing a delete you can: Or there's another version of this technique which can be even faster.


1 Answers

Calling DELETE FROM TableName will do the entire delete in one large transaction. This is expensive.

Here is another option which will delete rows in batches :

deleteMore: DELETE TOP(10000) Sales WHERE toDelete='1' IF @@ROWCOUNT != 0     goto deleteMore 
like image 101
Kevin Aenmey Avatar answered Oct 22 '22 15:10

Kevin Aenmey