Using MapMaker to create a cache

Question

I want to use MapMaker to create a map that caches large objects, which should be removed from the cache if there is not enough memory. This little demo program seems to work fine:

public class TestValue {
    private final int id;
    private final int[] data = new int[100000];

    public TestValue(int id) {
        this.id = id;
    }

    @Override
    protected void finalize() throws Throwable {
        super.finalize();
        System.out.println("finalized");
    }  
}  


public class Main {

    private ConcurrentMap<Integer, TestValue> cache;
    MemoryMXBean memoryBean;

    public Main() {
        cache = new MapMaker()
                .weakKeys()
                .softValues()
                .makeMap();
        memoryBean = ManagementFactory.getMemoryMXBean();
    }

    public void test() {
        int i = 0;
        while (true) {
            System.out.println("Etntries: " + cache.size() + " heap: "  
                + memoryBean.getHeapMemoryUsage() + " non-heap: "  
                + memoryBean.getNonHeapMemoryUsage());
            for (int j = 0; j < 10; j++) {
                i++;
                TestValue t = new TestValue(i);
                cache.put(i, t);
            }
            try {
                Thread.sleep(100);
            } catch (InterruptedException ex) {
            }
       }
    }

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) {
        Main m = new Main();
        m.test();
    }

}

However, when I do the same thing in my real application, entries are basically removed from the cache as soon as they are added. In my real application, I also use integers as keys, and the cached values are archive blocks read from disk that contains some data. As far as I understand, weak-references are garbage-collected as soon as they are no longer used, so this seems to make sense because the keys are weak references. If I create the map like this:

    data = new MapMaker()
            .softValues()
            .makeMap();

The entries are never garbage-collected and I get an out-of-memory error in my test program. The finalize method on the TestValue entries is never called. If I change the test method to the following:

public void test() {
    int i = 0;
    while (true) {
        for (final Entry<Integer, TestValue> entry :
            data.entrySet()) {
            if (entry.getValue() == null) {
                data.remove(entry.getKey());
            }
        }
        System.out.println("Etntries: " + data.size() + " heap: "
            + memoryBean.getHeapMemoryUsage() + " non-heap: "  
            + memoryBean.getNonHeapMemoryUsage());
        for (int j = 0; j < 10; j++) {
            i++;
            TestValue t = new TestValue(i);
            data.put(i, t);
        }
        try {
            Thread.sleep(100);
        } catch (InterruptedException ex) {
        }
    }
}

entries are removed from the cache and the finalizer on the TestValue objects is called, but after a while I also get an out-of-memory error.

So my question is: what is the right way to use MapMaker to create a map that can be used as a cache? Why does my test program not remove the entries as soon as possible if I use weakKeys? Is it possible to add a reference queue to the cache map?

Answer 1

There are a lot of things which might be going on, but with respect to your test program using soft values: you can get OutOfMemoryError even if you have SoftReferences which have not yet been garbage collected. That bears repeating: you can get an OutOfMemoryError even if you have SoftReferences which have not yet be cleared.

SoftReferences are a little weird, see http://jeremymanson.blogspot.com/2009/07/how-hotspot-decides-to-clear_07.html for a description of current mechanics. Likely in your test case, the GC just didn't have time to do two full GCs.

When you were using weakKeys, the CG cleared them right away, and didn't have to wait for a full GC pause. (b/c WeakReferences are collected aggressively.)

In my opinion, if you want a memory-sensitive cache with Integer keys, I'd think the following is appropriate:

data = new MapMaker().softValues().makeMap();

You can easily make a test program which throws OutOfMemoryError, but if your real application is somewhat well behaved, and doesn't get under too much pressure, you might be OK. SoftReferences are pretty hard to get right.

If you need to use System.gc() avoid out-of-memory, I would instead recommend you switch to an LRU map with a fixed max size (See the javadoc of java.util.LinkedHashMap for an example.) It's not concurrent, but I expect it's going to give you better throughput in the end than asking the system to do a full-pause garbage collection a bunch of extra times.

Oh, and a final note about integer keys and weakKeys(): MapMaker uses identity comparison for keys when using weak or soft keys, and that's pretty hard to do correctly. Witness the following:

Map<Integer,String> map = new MapMaker().weakKeys().makeMap();
Integer a = new Integer(1);
Integer b = new Integer(1);
Integer c = 1; //auto box
Integer d = 1; //auto box
map.put(a, "A");
map.put(b, "B");
map.put(c,"C");
map.put(d,"D");
map.size() // size is 3;

Good luck.

Answer 2

Weak keys seems like a mistake. Try using strong keys since they are integers.