rsync compression and how it works

Question

I am using rsync to backup a large amount of files (on a Ubuntu Linux server) to a cloud network service over WebDav. Because my network speed is my bottle-necking factor, I thought I would use rsync -z (compression) to decrease the file size and hopefully this would lessen the bottleneck.

Anyway, while researching the purpose and use of rsync with compression I don't seem to find an answer to exactly what it is doing.

Basically it seems that rsync compresses the data, transfers it, then decompresses the data. However, it doesn't make a lot of sense to me that rsync decompresses the data after it has been sent to the cloud as that would require the cloud service to decompress the data, which i don't think is happening.

My question is, is does rsync with compressing actually compress and decompress data that is sent over WebDav or FTP or some other network to network protocol? and if it does not help me, then in what scenario would the compression flag help me, local to local sync over USB 2.0 for example?

Answer 1

Where is the WebDAV folder mounted?

If it is on the local machine, you just get additional overhead with no performance gains, since the CPU has to do double work to compress and immediately decompress before writing to the mounted folder, which is then transferred to the remote server using the WebDAV protocol.

rsync does the compression in-transit using zlib. If you are not doing differential transfers, then scp offers faster performance due to the additional overhead in the rsync algorithm used to compare changes in file directory trees (although scp does not have a compress-in-transit option).

Compression is not always the answer though. If your network is the bottleneck then it does speed up the process, but if your CPU% is maxed out, it will just make the process slower.

rsync does not compress compressed filetypes (since the performance tradeoff ratio in compressing those types of files is very low) such as JPEG, LZO, LZMA/2, ZIP, GZIP, etc.

Answer 2

rsync since version 3.2.0 supports more than zlib:

--compress, -z
       turn on compression
--compress-choice=STR, --zc=STR
       choose compression from lz4 (fastest), zstd, zlibx, zlib (slowest), none
--compress-level=NUM, --zl=NUM
       zlib and zstd can be tuned with compression level
       zlib from 1 (lowest) to 9 (highest), default 6
       zstd from -131072 to 22, default 3