Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

In PostgreSQL 9.2, is archiving required for streaming replication?

Is it allowed and/or reasonable to configure a master PostgreSQL 9.2 server to NOT archive but to perform streaming replication. That is configured as shown below:

wal_level = hot_standby
archive_mode = off

Can the "slave" server (hot standby), be configured to archive WAL segments?

wal_level = hot_standby
hot_standby = on
archive_mode = on

This would allow the archiving network traffic on the master server to be cut in half (replication but not archiving). This seems reasonable and the documentation appears to support this configuration but I'd prefer a bit of reassurance that we have a good configuration.

like image 618
Dwayne Towell Avatar asked Oct 16 '13 15:10

Dwayne Towell


People also ask

How streaming replication works in PostgreSQL?

PostgreSQL streaming replication is based on transferring the WAL files from the primary to the target database. PostgreSQL streaming replication is implemented using a master-slave configuration. The master is known as the primary instance and handles the primary database and its operations.

What is archiving in PostgreSQL?

In PostgreSQL terms, copying out generated WAL files is called archiving, and getting the server to read in a WAL file and apply it is called restoring.

How does streaming replication work?

Streaming replication, a standard feature of PostgreSQL, allows the updated information on the primary server to be transferred to the standby server in real time, so that the databases of the primary server and standby server can be kept in sync.

What is the difference between streaming replication and logical replication?

Streaming replication was introduced for use with PostgreSQL v10. 0. Logical replication works by copying/replicating data objects and their changes based on their replication identity. In many cases, the data's identity is a primary key.


2 Answers

From documentation (strong added by myself):

If you use streaming replication without file-based continuous archiving, you have to set wal_keep_segments in the master to a value high enough to ensure that old WAL segments are not recycled too early, while the standby might still need them to catch up. If the standby falls behind too much, it needs to be reinitialized from a new base backup. If you set up a WAL archive that's accessible from the standby, wal_keep_segments is not required as the standby can always use the archive to catch up.

So, from my understanding, when you have too much transactions running, the slave could have some hard time to stay in sync. Especially if the master removes the WAL files before the slave really get what was inside. Without archive_mode on the master the WAL files could be deleted without leaving any way to get them back.

If you keep the WAL archiving in place and add the streaming upon a working hot-standby-with-archives structure this cannot happen as the slave could always access archived WAL and will get back the unsynced transactions as soon as the lower activity on the stream allows it. Without access to the archive the risk is clearly to loose your slave integrity after some really heavy stuff.

like image 179
regilero Avatar answered Oct 26 '22 10:10

regilero


I don't know if this is actual "official and certified", I also don't think it is recent, BUUT it comes form PostgreSQL Wiki.. (https://wiki.postgresql.org/wiki/Streaming_Replication)

Step 5, specifies interesting comments, which coincides with the answer of the post:

# To enable read-only queries on a standby server, wal_level must be set to
# "hot_standby". But you can choose "archive" if you never connect to the
# server in standby mode.
wal_level = hot_standby

# Set the maximum number of concurrent connections from the standby servers.
max_wal_senders = 5

# To prevent the primary server from removing the WAL segments required for
# the standby server before shipping them, set the minimum number of segments
# retained in the pg_xlog directory. At least wal_keep_segments should be
# larger than the number of segments generated between the beginning of
# online-backup and the startup of streaming replication. If you enable WAL
# archiving to an archive directory accessible from the standby, this may
# not be necessary.
wal_keep_segments = 32

# Enable WAL archiving on the primary to an archive directory accessible from
# the standby. If wal_keep_segments is a high enough number to retain the WAL
# segments required for the standby server, this is not necessary.
archive_mode    = on
archive_command = 'cp %p /path_to/archive/%f'
like image 31
zaratustra689 Avatar answered Oct 26 '22 09:10

zaratustra689