Compression with best ratio in Python?

Question

Which compression method in Python has the best compression ratio?

Is the commonly used zlib.compress() the best or are there some better options? I need to get the best compression ratio possible.

I am compresing strings and sending them over UDP. A typical string I compress has about 1,700,000 bytes.

Answer 1

I'm sure that there might be some more obscure formats with better compression, but lzma is the best, of those that are well supported. There are some python bindings here.

EDIT

Don't pick a format without testing, some algorithms do better depending on the data set.

Answer 2

The best compression algorithm definitely depends of the kind of data you are dealing with. Unless if you are working with a list of random numbers stored as a string (in which case no compression algorithm will work) knowing the kind of data usually allows to apply much better algorithms than general purpose ones (see other answers for good ready to use general compression algorithms).

If you are dealing with images you should definitely choose a lossy compression format (ie: pixel aware) preferably to any lossless one. That will give you much better results. Recompressing with a lossless format over a lossy one is a loss of time.

I would search through PIL to see what I can use. Something like converting image to jpeg with a compression ratio compatible with researched quality before sending should be very efficient.

You should also be very cautious if using UDP, it can lose some packets, and most compression format are very sensible to missing parts of file. OK. That can be managed at application level.