Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Fast decompression algorithms

I'm looking for compression/deompression algorithms that can give decent compression 2-4x on regular english text and yet I can decompress this data almost as fast as I can get it out of main memory (~10Gbps). Whats the current state of the art in terms of fast decompression algorithms (perhaps vectorized code that uses multiple cores)

In particular, I'm looking at this paper Fast Integer compression using SIMD instructions and wondering if similar algorithms have been used in any system.

like image 648
Amit Prakash Avatar asked Dec 03 '22 03:12

Amit Prakash


1 Answers

Look at LZO and lz4. Try them on your data and see how they perform.

like image 170
Mark Adler Avatar answered Dec 22 '22 18:12

Mark Adler