Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

c++ what in-memory compression library?

I already googled for in-memory compression and found quite a few libraries that offert this functionality. the zlib seems to be widely used - but it also seems to be quite old. I'm asking here whether there are newer, better alternatives.

The data i want to compress in-memory are memorypools with size of a few megabytes (2-16 MB) and each of those blocks contains data of two different structs as well as some arrays of pointers. inside the blocks, there's no particular order for the structs and the arrays, they are just allocated after another when the application needs to create such an element.

What compression lib would you suggest for this? compression and decompression performance (both) are more important than compression quality.

Also - for compression reasons - would it be better to have separate pools for the two different structs as well as the arrays, such that each datablock to be compressed only contains one kind of data?

This is the first time i intend to use in-memory compression and i know my question is maybe too general to give a good answer - but every hint is welcome!

thx!

like image 299
Mat Avatar asked Jan 02 '10 17:01

Mat


People also ask

What is memory in use compressed?

Memory compression is a memory management technique that reduces the size of inactive data in the random access memory (RAM) to free up unused space and allow more programs to run at once. It is designed to fully use the available physical memory and thereby increase the system's performance.

Should I turn on memory compression?

Now you know what memory compression is, why it is important, and how you can enable and, if you need to, disable it. However, we advise against disabling memory compression since it gives your RAM more room to play. It helps you avoid a majority of the problems that you can encounter when your physical memory is full.

Does memory compression affect performance?

The compressed memory feature is designed to improve the performance (responsiveness) of the system due to storing part of the memory pages in RAM in a compressed form. It helps to reduce the number of read / write requests to the memory pages in the slow (compared to RAM) paging file on a hard drive.


2 Answers

zlib is good. Proven, performant, and understood by many. It's what I'd use by default in a new system like what you describe. Its age should be seen as one of its greatest assets.

like image 140
John Zwinck Avatar answered Sep 30 '22 02:09

John Zwinck


For something more modern than zlib, libbzip2 might be worth a look. It provides a similar interface to zlib, for compatibility. In a lot of cases, it offers better compression, but at a performance cost.

For something faster than zlib (but which doesn't compress as well..) there's LZO.

like image 38
user242275 Avatar answered Sep 30 '22 00:09

user242275