How to serialize/deserialize a hashmap?

Question

I have a large hashmap containing millions of entries, and I want to persist it to disk, so that when it is read from the disk again, I don't have the overhead of inserting the key-value pairs back into the map again.

I am trying to use the cereal library to do this, but it appears that the HashMap datatype needs to derive Generic. Is there a way to do this?

Answer 1

You might be able to use stand-alone deriving to generate your own Generic instance for HashMap. You'll probably get a warning about orphan instances, but you also probably don't care :) Anyway, I haven't tried this, but it's probably worth a shot...

Answer 2

I am not sure if using Generics is a best shot at achieving high performance. My best bet would actually be writing your own instance for Serializable like this:

instance (Serializable a) => Serializable (HashMap a) where
  ...

To avoid creating orphan instances you can use newtype trick:

newtype SerializableHashMap a = SerializableHashMap { toHashMap :: HashMap a }
instance (Serializable a) => SerializableHashMap a where
  ...

The question is how to define ...?

There is no definite answer before you actually try and implement and benchmark possible solutions.

One possible solution is to use toList/fromList functions and store/read the size of the HashMap.

The other (which would be similar to using Generics) would be to write direct serialization based on internal HashMap structure. Given the fact that you dont really have the internals exported that would be a job for Generics only.