Java Garbage Collection Defragmentation

Question

I am getting a OutOfMemoryError java heap exception when just allocating and deallocating a byte array, even though there is enough free memory for the allocation. There is a short log below. I know from running on a different JVM, that the problem is due to fragmented memory and that the largest free block is only about 6MB. I thought that the java (oracle) JVM should take care of fragmented memory.

This is my test:

• Set up the java JVM with params -Xms10M -Xmx10M
• I allocate a byte array of say 90percent of the JVM Memory.
• Set this byte array then to null, then try to re-allocate the byte array of 90percent of the JVM memory.
• We see from the logs that JVM memory has reset to the full amount, yet we cant allocate the same amount of memory.
• Hence the memory must be fragmented?

Here are my details:

Total Free Memory: 9881728  
Attempting to Allocate Array of size: 6917209  
Freeing Allocated Array of size: 6917209  
Total Free Memory: 9881728  
Attempting to Allocate Array of size: 6917209  
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

The other JVM that I am running is CEE-J by Skelmir, I am running on a low memory device so I cannot set the MaxPermSize. It is interesting that the java heap is split into different areas, can these areas change size after garbage collection ?I have run another tests where I, First create a byte array for about 95 percent of the memory, then release the array again, then I create a load of arrays of objects which contain hashmaps, each with one entry, i clear the array and maps, then reallocate the same arrays again.

I check the memory each time i allocate and free. After doing this say 10 times, i free the memory for the last time, Then I check the memory, see that I have the full JVM allocation, I then try to allocate the 95% of the JVM allocation again and get the same JVM out of memory exception, even though it says I have enough memory to allocate?

public static void main(String[] args) throws InterruptedException {
        Runtime s_runtime = Runtime.getRuntime ();
        long used_memory = s_runtime.totalMemory () - s_runtime.freeMemory ();
        long start_memory = used_memory;
        long freeMemory = s_runtime.freeMemory();




        Integer numSubMemories = 1;
        Integer numMapEntries = 1;
        Integer numIterations = 1;
        float percentMem = (float) 1.0;
        if(args.length>0){
            if(args[0]!=null){
                String subEntries = args[0];
                if(subEntries.length()>0){
                    numSubMemories = Integer.parseInt(subEntries);
                }
            }
            if(args[1]!=null){
                String mapEntries = args[1];
                if(mapEntries.length()>0){
                    numMapEntries = Integer.parseInt(mapEntries);
                }
            }
            if(args[2]!=null){
                String numIts = args[2];
                if(numIts.length()>0){
                    numIterations = Integer.parseInt(numIts);
                }
            }
            if(args[3]!=null){
                String percentMemArg = args[3];
                if(percentMemArg.length()>0){
                    percentMem = Float.parseFloat(percentMemArg);
                }
            }
        }

        System.out.println("Total Free Memory: " + freeMemory);
        int numBytes = (int) (freeMemory * percentMem);
        System.out.println("Attempting to Allocate Array of size: " + numBytes);
        byte[] mbyteArray = new byte[numBytes];
        System.out.println("Freeing Allocated Array of size: " + numBytes);
        mbyteArray = null;



        TestMemory myMemory = new TestMemory(numSubMemories,numMapEntries);
        used_memory = s_runtime.totalMemory () - s_runtime.freeMemory ();
        System.out.println("Used Memory: " + used_memory);
        long after_allocation_memory = used_memory;
        for(int i=0;i<numIterations;i++){
            System.out.println("\t\t--Iteration: "+ i + " about to release Memory");
            myMemory.releaseMemoryArray(numSubMemories);

            System.out.println("\t\t--Iteration: "+ i + " about to reallocate Memory");
            myMemory.reAllocateMemoryArray(numSubMemories,numMapEntries);
            used_memory = s_runtime.totalMemory () - s_runtime.freeMemory ();
            System.out.println("\t\t--Used Memory: " + used_memory);
            System.out.println("\t\t--Max Memory: " + s_runtime.maxMemory());
            System.out.println("\t\t--Free Memory: " + s_runtime.freeMemory());
            System.gc();
            System.out.flush();
        }
        System.out.println("Releasing the Arrays for the last time");
         myMemory.releaseMemoryArray(numSubMemories);
         System.gc();
         freeMemory = s_runtime.freeMemory();
         //numBytes = (int)(freeMemory*0.95);
         System.out.println("Attempting to Allocate Array of size: " + numBytes);
         System.out.println("Total Free Memory: " + freeMemory);
         mbyteArray = new byte[numBytes];
        long end_memory = used_memory;
        //System.gc();
        Thread.sleep(6000);
        used_memory = s_runtime.totalMemory () - s_runtime.freeMemory ();

        System.out.println("------Results---------");
        System.out.println("        Start Memory: " + start_memory);
        System.out.println("        After Memory: " + after_allocation_memory);
        System.out.println("         End  Memory: " + end_memory);
        System.out.println("End Release   Memory: " + used_memory);

    }

Output Log from the second test. Total Free Memory: 9881728 Attempting to Allocate Array of size: 6917209 Freeing Allocated Array of size: 6917209 Used Memory: 7519744 -Iteration: 0 about to release Memory -Iteration: 0 about to reallocate Memory -Used Memory: 7890928 -Max Memory: 10092544 -Free Memory: 2147808 -Iteration: 1 about to release Memory -Iteration: 1 about to reallocate Memory -Used Memory: 930952 -Max Memory: 10092544 -Free Memory: 9107792 -Iteration: 2 about to release Memory -Iteration: 2 about to reallocate Memory -Used Memory: 1181832 -Max Memory: 10092544 -Free Memory: 8856912 -Iteration: 3 about to release Memory -Iteration: 3 about to reallocate Memory -Used Memory: 1190552 -Max Memory: 10092544 -Free Memory: 8848192 -Iteration: 4 about to release Memory -Iteration: 4 about to reallocate Memory -Used Memory: 902408 -Max Memory: 10092544 -Free Memory: 9190136 -Iteration: 5 about to release Memory -Iteration: 5 about to reallocate Memory -Used Memory: 902408 -Max Memory: 10092544 -Free Memory: 9190136 -Iteration: 6 about to release Memory -Iteration: 6 about to reallocate Memory -Used Memory: 902408 -Max Memory: 10092544 -Free Memory: 9190136 -Iteration: 7 about to release Memory -Iteration: 7 about to reallocate Memory -Used Memory: 902408 -Max Memory: 10092544 -Free Memory: 9190136 -Iteration: 8 about to release Memory -Iteration: 8 about to reallocate Memory -Used Memory: 902408 -Max Memory: 10092544 -Free Memory: 9190136 -Iteration: 9 about to release Memory -Iteration: 9 about to reallocate Memory -Used Memory: 902408 -Max Memory: 10092544 -Free Memory: 9190136 Releasing the Arrays for the last time Attempting to Allocate Array of size: 6917209 Total Free Memory: 9956688 Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

Answer 1

The Java heap is split into a number of areas. A single allocation cannot span areas. There are various -XX options in the Oracle JRE [I'll let some one else point to the details] that allow configuration of allocations. You could set the eden and survivor areas to be very small. Another issue if insufficient memory is available after a collection, you might still get an OOME to prevent thrashing (again more -XX options to control that).

Answer 2

Add -XX:MaxPermSize argument to provide adequate space to Heap.

-XX:MaxPermSize=512m