Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

postgresql shared memory settings

Hi all!

We have at present the following parameters related to shared memory:

postgres

shared_buffers = 7GB
max_connections = 1 500
max_locks_per_transaction = 1 024
max_prepared_transactions = 0 (not set)

system

SHMALL = 2 097 152
SHMMAX = 17 670 512 640
SHMMNI = 4096

The amount of RAM is 24 693 176k

We need to increase max_connections to 3 000. When we tried to do this, we got an error

[1-1] FATAL:  could not create shared memory segment: No space left on device
[2-1] DETAIL:  Failed system call was shmget(key=5432001, size=8964661248, 03600)
[3-1] HINT:  This error does *not* mean that you have run out of disk space.
It occurs either if all available shared memory IDs have been taken, in which
case you need to raise the SHMMNI parameter in your kernel, or because the
system's overall limit for shared memory has been reached.  If you cannot
increase the shared memory limit, reduce PostgreSQL's shared memory request
(currently 8964661248 bytes), perhaps by reducing shared_buffers or
max_connections.
The PostgreSQL documentation contains more information about shared memory
configuration.

The hint proposes to increase SHMMNI kernel parameter, but I'm not sure about how much to add :) Also, I believe that all these parameters correlate somehow, so do we need to change any other parameters accordingly?

Thanks in advance,

Alexander

like image 715
shutyaev Avatar asked Sep 27 '12 08:09

shutyaev


People also ask

What is shared memory in PostgreSQL?

Shared Memory: It is allocated by the PostgreSQL server when it is started, and it is used by all the processes. It is divided into sub-areas: Shared buffer pool: Where PostgreSQL loads pages with tables and indexes from disk, to work directly from memory, reducing the disk access.

How do I allocate memory to PostgreSQL?

shared_buffers (integer) The shared_buffers parameter determines how much memory is dedicated to the server for caching data. The value should be set to 15% to 25% of the machine's total RAM. For example: if your machine's RAM size is 32 GB, then the recommended value for shared_buffers is 8 GB.

What should Work_mem be set to?

The default value is 4 MB, which is good enough for most queries. If sorting requires more than the allocated work_mem , it leads to physical I/O operation, namely, having to write to temporary files on disk, which is expensive and degrades the performance of the query.

Does PostgreSQL use RAM?

There are some workloads where even larger settings for shared_buffers are effective, but given the way PostgreSQL also relies on the operating system cache, it's unlikely you'll find using more than 40% of RAM to work better than a smaller amount.


1 Answers

Increasing SHMMNI will not help, it's the second part of the hint that matters here.

Get your system's page size with the shell command getconf PAGE_SIZE.
Usually it's 4096. Multiply that by SHMALL.

In your case that should be 2097152 * 4096 = 8589934592, which is exactly 8Gb. That's your current maximum shared memory, independently of SHMMNI.

PostgreSQL error message indicates that it needs a bit more than that.

Conclusion: increase SHMALL.

like image 118
Daniel Vérité Avatar answered Oct 19 '22 03:10

Daniel Vérité