Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Java Multimap<String,String> with Trove

I have a pretty large google Multimap<String,String> and was looking into ways to reduce the memory usage. In all of the examples I can find people are doing something like:

Multimaps.newSetMultimap(
TDecorators.wrap(new TIntObjectHashMap<Collection<Integer>>()),
new Supplier<Set<Integer>>() {
public Set<Integer> get() {
  return TDecorators.wrap(new TIntHashSet());
}
});

which works for a Multimap <Integer,Integer>, is it possible to use Trove to wrap a <String,String>?

Incase anyone is interested in the future I went with http://code.google.com/p/jdbm2/ to write the hash map to the filesystem.

like image 820
chrstahl89 Avatar asked Jan 13 '23 20:01

chrstahl89


1 Answers

Guava's Multimaps are backed by standard JDK Collections which aren't optimized for memory usage. For example, ArrayListMultimap<K, V> is backed by HashMap<K, ArrayList<V>> and HashMultimap<K, V> is backed by HashMap<K, HashSet<V>>.

Eclipse Collections (formerly GS Collections) has Multimaps backed by its own container types, UnifiedMap and UnifiedSet. UnifiedMap uses half the memory of HashMap and UnifiedSet uses a quarter the memory of HashSet. The memory benefits you'll see will depend on whether you use a FastListMultimap or a UnifiedSetMultimap.

More detailed memory comparisons are available here.

Note: I am a committer for Eclipse Collections.

like image 104
Craig P. Motlin Avatar answered Jan 17 '23 13:01

Craig P. Motlin