MPI: is there mpi libraries capable of message compression?

Question

Sometimes MPI is used to send low-entropy data in messages. So it can be useful to try to compress messages before sending it. I know that MPI can work on very fast networks (10 Gbit/s and more), but many MPI programs are used with cheap network like 0,1G or 1Gbit/s Ethernet and with cheap (slow, low bisection) network switch. There is a very fast Snappy (wikipedia) compression algorithm, which has

Compression speed is 250 MB/s and decompression speed is 500 MB/s

so on compressible data and slow network it will give some speedup.

Is there any MPI library which can compress MPI messages (at layer of MPI; not the compression of ip packets like in PPP).

MPI messages are also structured, so there can be some special method, like compression of exponent part in array of double.

PS: There is also LZ4 compression method with comparable speed

Answer 1

I won't swear that there's none out there, but there's none in common use.

There's a couple of reason's why it's not common:

MPI is often used for sending lots of floating point data which is hard (but not impossible) to compress well, and often has relatively high entropy after a while.

In addition, MPI users are often as concerned with latency as bandwidth, and adding a compression/decompression step into the message-passing critical path wouldn't be attractive to those users.

Finally some operations (like reduction collectives, or scatter gather) would be very hard to implement efficiently with compression.

However, you sound like your use case could benefit from this for point-to-point communications, so there's no reason why you couldn't do it yourself. If you were going to send a message of size N and the receiver expected it then:

• sender calls compression routine, receives buffer and new length M;
• if M >= N, send the original data, with an initial byte of say 0, as N+1 bytes to the receiver
• otherwise sends an initial byte of 1 + compressed data
• receiver receives data into length N+1 buffer
• if first byte is 1, calls MPI_Get_count to determine amount of data received, calls decompression routine
• otherwises uses uncompressed data

I can't give you much guidance as to the compresion routines, but it does look like people have tried this before, eg http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.91.7936 .