Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Automatic partitioning by day - PostgreSQL

I would like to do a daily partitions. I know with oracle is something like this.

CREATE TABLE "PUBLIC"."TEST"     
(   
"ID" NUMBER(38,0) NOT NULL ENABLE, 
"SOME_FIELD" VARCHAR2(20 BYTE) NOT NULL ENABLE,     
"ANOTHER_FIELD" VARCHAR2(36 BYTE) NOT NULL ENABLE,    
TABLESPACE "PUBLIC"."TEST_DATA" 
PARTITION BY RANGE ("TEST_DATE") INTERVAL (NUMTODSINTERVAL(1,'DAY'))  
(PARTITION "TEST_P1"  
VALUES LESS THAN (TIMESTAMP' 2019-01-01 00:00:00')   TABLESPACE "TEST_DATA" );

What about PostgreSQL?

NEW EDIT: SAMPLE SCRIPT:

The script which will maintain first 15 days data in one table say "p1" and remaining days data in another partition.

1- Creating automatic partion depends on the date range of insert command

2- In script i have also mentioned that how we can add index on the required column's.

3- Data from date range from 1st to 14th will be added in partition "p1" and remaining will be added in partition "p2".

Sample Script :

    CREATE TABLE measurement (
        city_id         int not null,
        logdate         date not null,
        peaktemp        int,
        unitsales       int
    );

    CREATE OR REPLACE FUNCTION new_partition_creator() RETURNS trigger AS
  $BODY$
    DECLARE
      partition_date TEXT;
      partition TEXT;
      partition_day int;
      startdate date;
      enddate date;
    BEGIN
      partition_day := to_char(NEW.logdate,'DD');
      partition_date := to_char(NEW.logdate,'YYYY_MM');

         IF partition_day < 15 THEN
      partition := TG_RELNAME || '_' || partition_date || '_p1';
      startdate := to_char(NEW.logdate,'YYYY-MM-01');
      enddate := date_trunc('MONTH', NEW.logdate) + INTERVAL '1 MONTH - 1 day';
      ELSE 
      partition := TG_RELNAME || '_' || partition_date || '_p2';
      startdate := to_char(NEW.logdate,'YYYY-MM-15');
      enddate := date_trunc('MONTH', NEW.logdate) + INTERVAL '1 MONTH - 1 day';
      END IF;

      IF NOT EXISTS(SELECT relname FROM pg_class WHERE relname=partition) THEN
        RAISE NOTICE 'A partition has been created %',partition;
        EXECUTE 'CREATE TABLE ' || partition || ' ( CHECK ( logdate >= DATE ''' || startdate || '''  AND logdate <=  DATE ''' ||  enddate || ''' )) INHERITS (' || TG_RELNAME || ');';
        EXECUTE 'CREATE INDEX ' || partition || '_logdate ON '  || partition || '(logdate)';
        EXECUTE 'ALTER TABLE ' || partition || ' add primary key(city_id);';       
        END IF;
        EXECUTE 'INSERT INTO ' || partition || ' SELECT(' || TG_RELNAME || ' ' || quote_literal(NEW) || ').* RETURNING city_id;';
      RETURN NULL;
    END;
  $BODY$
LANGUAGE plpgsql VOLATILE
COST 100; 


    CREATE TRIGGER testing_partition_insert_trigger BEFORE INSERT ON measurement FOR EACH ROW EXECUTE PROCEDURE new_partition_creator();


    postgres=# insert into measurement values(1,'2017-10-11',10,10);
    NOTICE:  A partition has been created measurement_2017_10_p1
    INSERT 0 0
like image 512
R. Pereira Avatar asked Apr 11 '19 22:04

R. Pereira


People also ask

How do I automate partitions in PostgreSQL?

Automatic partition creation for time-triggered partitioning You can use the operating system scheduler ( cron on Unix-like systems or “Task Scheduler” on Windows). The job would connect to the database using psql and run the appropriate CREATE TABLE and ALTER TABLE ... ATTACH PARTITION statements.

Does PostgreSQL support partitioning?

PostgreSQL supports basic table partitioning. This section describes why and how to implement partitioning as part of your database design.

How many Postgres partitions is too many?

Postgres 10 – It can handle few hundred partitioned tables before performance degradation. Postgres 11 – It can handle up to 2-3K partitioned tables before performance degradation. In my testing, using 24K partitions caused an out of memory issue.

What is default partition in Postgres?

The PostgreSQL 11 DEFAULT partition feature stores tuples that don't map to any other partition. Prior to PostgreSQL 11, these rows would error out. A row that is not mapped to any partition table would be inserted in the default partition.


1 Answers

You can use extension pg_partman for automatic partition creation. https://github.com/pgpartman/pg_partman

or you can even use scheduler pg_agent where you will execute a procedure every day at say 18:00:00 to create next days partition.

like image 66
Amol Tarte Avatar answered Nov 19 '22 19:11

Amol Tarte