Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Text compression in PostgreSQL

Tags:

postgresql

I know in SQL we can compress the text field like the following:

CREATE TABLE TableName (FieldName CHARACTER(255) WITH COMPRESSION); 

I want to know how to achieve the text compression in Postgres.

like image 343
karthi_ms Avatar asked Sep 27 '10 05:09

karthi_ms


People also ask

Does Postgres support compression?

PostgreSQL uses a fixed page size (commonly 8 kB), and does not allow tuples to span multiple pages. Therefore, it is not possible to store very large field values directly. To overcome this limitation, large field values are compressed and/or broken up into multiple physical rows.

Does Postgres automatically compress large text values?

In a nutshell, when a value is wider than 2 kB (set by TOAST_TUPLE_THRESHOLD ) and the default EXTENDED storage method is used, compression automatically kicks in. This information led me to ask the following questions: How much compression is achieved on TEXT ?

Does Postgres compress JSON?

ZSON is a PostgreSQL extension for transparent JSONB compression. Compression is based on a shared dictionary of strings most frequently used in specific JSONB documents (not only keys, but also values, array elements, etc). In some cases ZSON can save half of your disk space and give you about 10% more TPS.

What is toast in PostgreSQL?

TOAST is a mechanism PostgreSQL uses to keep physical data rows from exceeding the size of a data block (typically 8KB). PostgreSQL does not support physical rows that cross block boundaries, so the block size is a hard upper limit on row size.


1 Answers

Compression is enabled by default for all string types, you don't have to tell the database to do it. Check the manual about TOAST

  • PLAIN prevents either compression or out-of-line storage; furthermore it disables use of single-byte headers for varlena types. This is the only possible strategy for columns of non-TOAST-able data types.
  • EXTENDED allows both compression and out-of-line storage. This is the default for most TOAST-able data types. Compression will be attempted first, then out-of-line storage if the row is still too big.
  • EXTERNAL allows out-of-line storage but not compression. Use of EXTERNAL will make substring operations on wide text and bytea columns faster (at the penalty of increased storage space) because these operations are optimized to fetch only the required parts of the out-of-line value when it is not compressed.
  • MAIN allows compression but not out-of-line storage. (Actually, out-of-line storage will still be performed for such columns, but only as a last resort when there is no other way to make the row small enough to fit on a page.)
like image 184
Frank Heikens Avatar answered Nov 01 '22 04:11

Frank Heikens