Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Should I partition/subpartition my table?

Case

  1. The system has dispositives, basically formed by id, type, name.
  2. I may have N dispositives.
  3. I have a table to store a log of all dispositives. This is the biggest table in the system. (Now counting 100 mi records)
  4. The log table has: id_dispositive, date, status.

Problem

Obviously this huge data it's causing performance issues. I must store at least two months log values..

Today I have this dispositives by type:

TYPE     COUNT
---------------
1         78956  
2         125161
3         13213
4         6112
5         25426
6         12314
7         1241
8         622

I also have some business logic to feed this log table. Anyways, here is my thoughts:

My "solution"

I was thinking on partition this log table. Here is the questions:

  1. Does it worth it?
  2. Should I partition by type?
  3. Should I partition by type AND subpartition by DATE?
  4. Can I do it programmatically?

As this types are dynamics, users can delete/insert new ones.

So if they delete some type, I must delete all logs of that type. As if they insert some type, I must register(store) all logs of that type.

EDIT according Francesco Serra's answer

I'm trying to create the table like this:

create table log(
       id_dispositive    number,
       type       number,
       date_verification    date,
       status  number
)
partition by range (date_verification) 
subpartition by list (type)  
subpartition TEMPLATE (
    SUBPARTITION type1 VALUES (1),
    SUBPARTITION type2 VALUES (2),
    SUBPARTITION type3 VALUES (3),
    SUBPARTITION type4 VALUES (4)
)(                           
interval (numtoyminterval(1,'MONTH'))(
   partition p0816 values less than (to_date('01/09/2016','dd/mm/yyyy'))
));

and i'm getting:

ORA-14004: PARTITION keyword not found.
like image 612
Marllon Nasser Avatar asked Nov 08 '22 11:11

Marllon Nasser


1 Answers

You can try in this way: use INTERVAL to tell oracle to create automatic partitions. You must define column (number or date) and an interval (in my example 1 month). Oracle will put in the same partition all row in the same interval (in this case in the same month). If the partition doesn't exist will be created.

create table log(
       id_dispositive    number,
       date    date,
       status  number,
       type    number
)
partition by range (date)                              
interval (numtoyminterval(1,'MONTH'))(
   partition p0701 values less than (to_date('2007-02-01','yyyy-mm-dd'))
);

The same can be done also with type column. More information: http://www.oracle.com/technetwork/articles/sql/11g-partitioning-084209.html.

Every partition can be subpartitioned using TEMPLATE key.

create table log(
       id_dispositive    number,
       date    date,
       status  number,
       type    number
)
partition by range (date) interval (numtoyminterval(1,'MONTH'))
subpartition by list (type)  
subpartition TEMPLATE (
    SUBPARTITION types1 VALUES (1, 2) TABLESPACE tbs_1,
    SUBPARTITION types2 VALUES (3, 4) TABLESPACE tbs_1
)                          
(
   partition p0701 values less than (to_date('2007-02-01','yyyy-mm-dd'))
);

In this case you can't create an automatic subpartition, if a new type will be add you have to run an alter table statment. Here more informations; https://docs.oracle.com/cd/B28359_01/server.111/b32024/part_admin.htm#i1006655.

In your example:

create table prova_log(
       id_dispositive    number,
       type       number,
       date_verification    date,
       status  number
)
partition by range (date_verification) interval (numtoyminterval(1,'MONTH'))
subpartition by list (type)  
subpartition TEMPLATE (
    SUBPARTITION type1 VALUES (1),
    SUBPARTITION type2 VALUES (2),
    SUBPARTITION type3 VALUES (3),
    SUBPARTITION type4 VALUES (4)
)                          
(
   partition p0816 values less than (to_date('01/09/2016','dd/mm/yyyy'))
);

If you try to insert:

insert into prova_log values (1,1,TO_DATE('10/10/2016','dd/mm/yyyy'),1);

you will see 2 partition on your table.

Now I've tested it!

like image 50
Francesco Serra Avatar answered Nov 15 '22 09:11

Francesco Serra