Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Implementing in-memory compression for objects in Java

We have this use case where we would like to compress and store objects (in-memory) and decompress them as and when required.

The data we want to compress is quite varied, from float vectors to strings to dates.

Can someone suggest any good compression technique to do this ?

We are looking at ease of compression and speed of decompression as the most important factors.

Thanks.

like image 732
Kichu Avatar asked May 09 '11 08:05

Kichu


People also ask

How do you compress an object in Java?

Java object compression is done using the GZIPOutputStream class (this class implements a stream filter for writing compressed data in the GZIP file format) and passes it to the ObjectOutputStream class (this class extends OutputStream and implements ObjectOutput, ObjectStreamConstants) to write the object into an ...

How do you compress an object?

If you need to compress arbitrary objects, a possible approach is to serialize the object into a byte array, and then use e.g. the DEFLATE algorithm (the one used by GZIP) to compress it. When you need the object, you can decompress and deserialize it.

How are Java objects stored in memory in Java?

A stack and a heap are used for memory allocation in Java. However, the stack is used for primitive data types, temporary variables, object addresses etc. The heap is used for storing objects in memory.

Where Java objects are stored in in memory?

Heap space is used for the dynamic memory allocation of Java objects and JRE classes at runtime. New objects are always created in heap space, and the references to these objects are stored in stack memory.


1 Answers

If you want to compress instances of MyObject you could have it implement Serializable and then stream the objects into a compressed byte array, like so:

ByteArrayOutputStream baos = new ByteArrayOutputStream(); GZIPOutputStream gzipOut = new GZIPOutputStream(baos); ObjectOutputStream objectOut = new ObjectOutputStream(gzipOut); objectOut.writeObject(myObj1); objectOut.writeObject(myObj2); objectOut.close(); byte[] bytes = baos.toByteArray(); 

Then to uncompress your byte[] back into the objects:

ByteArrayInputStream bais = new ByteArrayInputStream(bytes); GZIPInputStream gzipIn = new GZIPInputStream(bais); ObjectInputStream objectIn = new ObjectInputStream(gzipIn); MyObject myObj1 = (MyObject) objectIn.readObject(); MyObject myObj2 = (MyObject) objectIn.readObject(); objectIn.close(); 
like image 196
WhiteFang34 Avatar answered Sep 22 '22 10:09

WhiteFang34