Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to periodically archive data in MySQL

Tags:

sql

mysql

I got this scenario :

I got 2 specific columns which are called cs_start_time(Type:Datetime) and cs_time_length(Type:int (seconds)) in my table.

Now I want to delete these rows automatically when cs_start_time + cs_time_length < NOW() (Current Datetime) and Insert them into another table (of course it should be before deleting these rows).

Or I handle it when someone do a select query on these rows (Trigger alike)

Is this even possible with mysql ? (Im a newbie in mysql)

like image 655
Ahmet Kazaman Avatar asked May 02 '16 05:05

Ahmet Kazaman


People also ask

What is PT Archiver?

Pt-archiver uses the index to select records from the table. The index is used to optimize repeated accesses to the table. Pt-archiver remembers the last row it retrieves from each SELECT statement, and uses it to construct a WHERE clause.

Can we store list in MySQL?

You can store it as comma separated values. Basically mysql only storing data as string or blob format. So list can be stored as csv values. You can use ElementCollection annotation and store it.


1 Answers

You can use MySQL event scheduler for this purpose:

I've simulated your scenario creating the main table where data initially stay. Later the expired data are archived in a table named archiveTable through an event scheduler.

  • Main table structure and data:

-- ----------------------------
-- Table structure for `maintable`
-- ----------------------------
DROP TABLE IF EXISTS `maintable`;
CREATE TABLE `maintable` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `cs_start_time` datetime NOT NULL,
  `cs_time_length` int(11) NOT NULL,
  PRIMARY KEY (`id`)
);

-- ----------------------------
-- Records of maintable
-- ----------------------------
INSERT INTO `maintable` VALUES ('1', '2016-05-01 12:00:00', '10');
INSERT INTO `maintable` VALUES ('2', '2016-05-02 12:00:00', '5');
INSERT INTO `maintable` VALUES ('3', '2016-05-03 12:00:00', '15');

  • Archive table structure and data:

DROP TABLE IF EXISTS `archivetable`;
CREATE TABLE `archivetable` (
  `id` int(11) NOT NULL,
  `cs_start_time` datetime NOT NULL,
  `cs_time_length` int(11) NOT NULL
);

  • Set event scheduler ON first

SET GLOBAL event_scheduler = ON;

  • Create the event:

    DROP EVENT IF EXISTS `archiveEvent`;
    delimiter $$
    CREATE EVENT `archiveEvent` ON SCHEDULE EVERY 1 MINUTE STARTS '2016-05-02 00:00:00' ON COMPLETION PRESERVE ENABLE DO

    BEGIN
        INSERT INTO archivetable (
            id,
            cs_start_time,
            cs_time_length
        ) SELECT
            MT.id,
            MT.cs_start_time,
            MT.cs_time_length
        FROM
            maintable MT
        WHERE
            MT.cs_start_time + INTERVAL MT.cs_time_length SECOND < NOW() ; 

        DELETE
        FROM
            maintable
        WHERE
            cs_start_time + INTERVAL cs_time_length SECOND < NOW() ;
        END$$

    delimiter ;

Note: Look the event start time is set to 2016-05-02 00:00:00. After that the event will be scheduled in every one minute interval. You can change the schedule time in any interval unit as you like.

Suggestion:

Quoting from my comment:

You can use mysql event scheduler for this purpose. But I would suggest you can filter out these entries in your select query. What's the big deal of deleting those entries whereas you can easily bypass these in your select query? You have to check every second through the scheduler whether they meet the condition to be deleted or not;

like image 66
1000111 Avatar answered Sep 30 '22 10:09

1000111