Can't consume entire memory

Question

I am trying to write a program which should consume memory of a specific size. An issue I am wondering of is that I am getting outOfMemory exception when there is actually a free space in the heap.

Here is the code:

import java.util.Vector;
import java.lang.*;

public class MemoryEater1 {
  public static void main(String[] args) {
    try {
        long mb = Long.valueOf(args[0]);
        Vector v = new Vector();
        Runtime rt = Runtime.getRuntime();
        while (true) {
            if (v.size() > 0) {
                if (((long) v.size())*100 <  mb) {
                    System.out.println("total memory: " + rt.totalMemory()/1024/1024);
                    System.out.println("max memory: " + rt.maxMemory()/1024/1024);
                    System.out.println("free memory: " + rt.freeMemory()/1024/1024);                        
                    System.out.println("Trying to add 100 mb");                 
                    //100mb
                    byte b[] = new byte[104857600];
                    v.add(b);
                }
            } else {
                //100mb
                byte b[] = new byte[104857600];
                v.add(b);
                System.out.println("Added 100 mb");             
            }
        }
    } catch (Exception e) {
        e.printStackTrace();
    }
  }
}

The command to start it:

 java -Xmx4096m MemoryEater1 3000

And the output:

total memory: 2867
max memory: 3641
free memory: 59
Trying to add 100 mb
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at MemoryEater1.main(MemoryEater1.java:18)

Well the difference between max memory and total memory is 774mb, which should be enough to consume 100mb more, but still there is the error, and even the machine resources are sufficient enough:

[user@machine ~]$ free -m
             total       used       free     shared    buffers     cached
Mem:         15950       3447      12502          0        210       2389
-/+ buffers/cache:        847      15102
Swap:         4031 1759218603       8941

Why can that be?

Answer 1

I don't think it's fragmentation as you only have one thread allocating memory and not reclaiming anything.

It's your particular garbage collector to blame, they manage memory differently resulting in more or less being unavailable for your app. You can find out which one is used by analyzing the output of java -XX:+PrintCommandLineFlags).

You can try to use G1 which manages memory differently.

java -Xmx4096m -XX:+UseG1GC MemoryEater1 3000

Or play with generation sizes e.g. -XX:NewSize and so on.

For more information read up VM options and anything on garbage collector algorithms e.g. [GC tuning]

Here's a quick illustration how splitting memory for different generations can make it unavailable (http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html).