Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Deleting data from a large table

Tags:

sql

I have a table with about 10 fields to store gps info for customers. Over time as we have added more customers that table has grown to about 14 million rows. As the gps data comes in a service constantly inserts a row into the table. 90% of the data is not revelent i.e. the customer does not care where the vehicle was 3 months ago, but the most recent data is used to generate tracking reports. My goal is to write a sql to perform a purge of the data that is older than a month.

Here is my problem I can NOT use TRUNCATE TABLE as I would lose everything? Yesterday I wrote a delete table statement with a where clause. When I ran it on a test system it locked up my table and the simulation gps inserts were intermittently failing. Also my transaction log grew to over 6GB as it attempted to log each delete.

My first thought was to delete the data a little at a time starting with the oldest first but I was wondering if there was a better way.

like image 877
Ron Skufca Avatar asked May 13 '09 16:05

Ron Skufca


People also ask

How do you remove unwanted records from a table?

When you are in Show Selected Records mode, CTRL+D deletes the currently highlighted (yellow) records in your selection, making it easy to delete a subset of the records in your current selection.


1 Answers

My 2 cents:

If you are using SQL 2005 and above, you can consider to partition your table based on the date field, so the table doesn't get locked when deleting old records.

Maybe, if you are in position of making dba decisions, you can temporarily change your log model to Simple, so it won't grow up too fast, it will still be growing, but the log won't be too detailed.

like image 55
Jhonny D. Cano -Leftware- Avatar answered Sep 23 '22 06:09

Jhonny D. Cano -Leftware-