Is there any chance that packing a large file with some simple algorithm enables me to read the data faster than from an uncompressed file (due to the hard drive being slower than uncompressing)? What kind of compression rate would I need to have? Can any fast compression algorithm do that?
Yes. That is often the case with deflate compression, used by zip, gzip, and zlib, when reading from hard drives with a typical compression factor of, say, four.
From SSDs, you may need to go to something with faster decompression. One you could try is lz4.
Your mileage may vary.
You could also try Density, its command line client "sharc" is benchmarked here.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With