Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to check if TOAST is working on a particular table in postgres

I have a table that contains two text fields which hold a lot of text. For some reason our table have started growing exponentially. I suspect that TOAST (compression for text fields in postgres) is not working automatically. In our table definition we have not defined any clause to force compression of these fields. Is there a way to check if compression is working on that table or not?

like image 406
jindal Avatar asked Apr 16 '14 21:04

jindal


People also ask

What is the toast mechanism in PostgreSQL?

TOAST is a mechanism PostgreSQL uses to keep physical data rows from exceeding the size of a data block (typically 8KB). Postgres does not support physical rows that cross block boundaries, so the block size is a hard upper limit on row size. To allow user tables to have rows wider than this, the TOAST mechanism breaks up wide field values into ...

Where are toast values stored in SQL Server?

If any of the columns of a table are TOAST -able, the table will have an associated TOAST table, whose OID is stored in the table's pg_class. reltoastrelid entry. On-disk TOAST ed values are kept in the TOAST table, as described in more detail below.

What data types are toasted in PostgreSQL?

All standard Postgres data types that could possibly have values wider than 2KB support being "TOASTed" in this way, and so do most potentially-wide extension data types. You can view the current TOAST options for a table by opening psql and running

How to check the associated toast table in PG_class?

You can check the associated toast table in pg_class using the following query: The table name will be pg_toast_$ (OID) where oid is the toast table oid, reltoastrelid of original table. The toast table is in pg_toast schema, so to query you need to use: The above query will print the toasted chunk_id, chunk_seq, chunk_data, if present.


1 Answers

From the docs . . .

If any of the columns of a table are TOAST-able, the table will have an associated TOAST table, whose OID is stored in the table's pg_class.reltoastrelid entry. Out-of-line TOASTed values are kept in the TOAST table, as described in more detail below.

So you can determine whether a TOAST table exists by querying the pg_class system catalog. This should get you close to what you're looking for.

select t1.oid, t1.relname, t1.relkind, t2.relkind, t2.relpages, t2.reltuples
from pg_class t1
inner join pg_class t2
on t1.reltoastrelid = t2.oid
where t1.relkind = 'r'
  and t2.relkind = 't';

In psql, you can use \d+. I'll use the pg_class system catalog as an example; you'd use your own table name.

sandbox=# \d+ pg_class
     Column     |   Type    | Modifiers | Storage  | Stats target | Description 
----------------+-----------+-----------+----------+--------------+-------------
 relname        | name      | not null  | plain    |              | 
 relnamespace   | oid       | not null  | plain    |              | 
 [snip]
 relacl         | aclitem[] |           | extended |              | 
 reloptions     | text[]    |           | extended |              | 

Where Storage is 'extended', PostgreSQL will try to reduce row size by compressing first, then by storing data out of line. Where Storage is 'main' (not shown), PostgreSQL will try to compress.

In your particular case, you might find it useful to monitor changes in size over time. You can use this query, and save the results for later analysis.

select table_catalog, table_schema, table_name, 
       pg_total_relation_size(table_catalog || '.' || table_schema|| '.' || table_name) as pg_total_relation_size,
       pg_relation_size(table_catalog || '.' || table_schema|| '.' || table_name) as pg_relation_size,
       pg_table_size(table_catalog || '.' || table_schema|| '.' || table_name) as pg_table_size
from information_schema.tables

PostgreSQL admin functions has details about what each function includes in its calculations.

like image 167
Mike Sherrill 'Cat Recall' Avatar answered Sep 21 '22 19:09

Mike Sherrill 'Cat Recall'