Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

StringBuilder out of memory error when working with large strings in java

I went from String test += str; where test grew exponentially large with thousands and thousands of characters. It took 45 minutes to run, probably from creating large strings and deleting garbage. I then staggered the input like this which brought it to 30 seconds.

This seems like the cheap way to do it, but it worked well:

  if (secondDump.length() > 50)
  {
     intermedDump = intermedDump + secondDump;
     secondDump = "";
  }      

  if (intermedDump.length() > 100)
  {
     thirdDump = thirdDump + intermedDump;
     intermedDump = "";
  }
  if (thirdDump.length() > 500)
  {
     fourthDump = fourthDump + thirdDump;
     thirdDump = "";
  }
  if (fourthDump.length() > 1000)
  {
     fifthDump = fifthDump + fourthDump;
     fourthDump = "";
  }
  //with just this and not sixth.  Runtime>>>> : 77343
  if (fifthDump.length() > 5000)
  {
     sixthDump = sixthDump + fifthDump;
     fifthDump = "";
  }
  //with just this.  Runtime>>>> : 35903Runtime>>>> : 33780
  if (sixthDump.length() > 10000)
  {
     fillerDump = fillerDump + sixthDump;
     sixthDump = "";
  }

I then discovered that StringBuilder exists, and I've been trying to use it since, replacing all string operations with it.

The problem is, I keep getting an java.lang.OutOfMemoryError with a java memory heap overflow. I think the string is just too long to store in memory as a StringBuilder object, because it makes about 1/50th of the progress that my previous code did before crashing with an out of memory error. It's only working with maybe under a thousand characters.

Why can a string hold the entire output and this can't come close? Also, if I append text to JTextPane, how much memory does that need? If I dump the StringBuilder contents to JTextpane and keep appending and clearing StringBuilder that doesn't seem to work either.

Here is the existing code. Page is just an object being passed:

protected void concatPlates(page PageA) throws IOException
{
   if (backend.isFirstPage == false)
   {
      frontend.fillOutputPane("\n                                 " +
         "                                               \n", PageA);
      frontend.fillOutputPane("                                 " +
         "                                               \n", PageA);
      frontend.fillOutputPane("                                 " +
         "                                               \n", PageA);
   }
   for (int i = 0; i < PLATELEN-1; i++)
   {

      if (arrLeftCol[i].length() == 0)
      {
         /////////////////////////////////////////////////////////
         /////////////////////////////////////////////////////////
         frontend.fillOutputPane(arrLeftCol[i].append(
           arrRightCol[i]));
      }
     else
     {
        PageA.tempStrBuff = new StringBuilder(arrLeftCol[i].substring(0,40));
        frontend.fillOutputPane(PageA.tempStrBuff.append(arrRightCol[i]));
     }
     arrLeftCol[i].append("");
     arrRightCol[i].append("");

     backend.isFirstPage = false;
  }
}


//this is the frontend class
public static void fillOutputPane(String s, page PageA)
{
   fillOutputPane(PageA.getStrBuf());
}
public static void fillOutputPane(StringBuilder stringBuild)
{
  try
  {
     str.append(stringBuild);
  }
  catch (java.lang.OutOfMemoryError e)
  {
     System.out.println((str.length() * 16) /8);
     //System.out.println(str);
     System.out.println("out of memory error");
     System.exit(0);
  }
}

Here is the error:

Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Unknown Source)
at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
at java.lang.AbstractStringBuilder.append(Unknown Source)
at java.lang.StringBuilder.append(Unknown Source)
at java.lang.StringBuilder.append(Unknown Source)
at backend.fill(backend.java:603)
at frontend$openL.actionPerformed(frontend.java:191)
at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
at javax.swing.AbstractButton.doClick(Unknown Source)
at javax.swing.plaf.basic.BasicMenuItemUI.doClick(Unknown Source)
at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(Unknown Source)
at java.awt.Component.processMouseEvent(Unknown Source)
at javax.swing.JComponent.processMouseEvent(Unknown Source)
at java.awt.Component.processEvent(Unknown Source)
at java.awt.Container.processEvent(Unknown Source)
at java.awt.Component.dispatchEventImpl(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Window.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
at java.awt.EventQueue.access$000(Unknown Source)
at java.awt.EventQueue$1.run(Unknown Source)
at java.awt.EventQueue$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)

I think this is what a stack trace is:

java.lang.Exception: Stack trace
at java.lang.Thread.dumpStack(Unknown Source)
at frontend.fillOutputPane(frontend.java:385)
at page.concatPlates(page.java:105)
at backend.setPlate(backend.java:77)
at backend.fill(backend.java:257)
at frontend$openL.actionPerformed(frontend.java:191)
at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
at javax.swing.AbstractButton.doClick(Unknown Source)
at javax.swing.plaf.basic.BasicMenuItemUI.doClick(Unknown Source)
at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(Unknown Source)
at java.awt.Component.processMouseEvent(Unknown Source)
at javax.swing.JComponent.processMouseEvent(Unknown Source)
at java.awt.Component.processEvent(Unknown Source)
at java.awt.Container.processEvent(Unknown Source)
at java.awt.Component.dispatchEventImpl(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Window.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
at java.awt.EventQueue.access$000(Unknown Source)
at java.awt.EventQueue$1.run(Unknown Source)
at java.awt.EventQueue$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source)
at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source)
at java.awt.EventQueue$2.run(Unknown Source)
at java.awt.EventQueue$2.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source)
at java.awt.EventQueue.dispatchEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)81240560
at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.run(Unknown Source)
like image 537
montag Avatar asked Dec 02 '11 16:12

montag


People also ask

What is the maximum size of StringBuilder in Java?

Here because of parameter as int the maximum capacity that a StringBuilder Class can is reach will be 2147483647 .

How big can a StringBuilder be?

The length is an int so it should hold up to 2GChar (4GB) assuming you have the memory. You are going to use "only" 600MB (300 million @ 2 bytes per character).

Is there a limit to String builder?

The maximum capacity for this implementation is Int32. MaxValue. However, this value is implementation-specific and might be different in other or later implementations. You can explicitly set the maximum capacity of a StringBuilder object by calling the StringBuilder(Int32, Int32) constructor.

Should you use StringBuilder to construct large text instead of just adding strings?

I think we should go with StringBuilder append approach. Reason being : The String concatenate will create a new string object each time (As String is immutable object) , so it will create 3 objects. With String builder only one object will created[StringBuilder is mutable] and the further string gets appended to it.


1 Answers

Apparently your application doen't have enough memory for complete the operation. So you need to specify memory flags to your virtual machine. You can try the following:

 java -Xms256m -Xmx512m YourApp

Where:

  • Xms minimun memory allocated by your program at startup (in the example 256 MB)
  • Xmx maximun memory allocated by your program if it need more (in the example 512 MB)
like image 133
Ernesto Campohermoso Avatar answered Sep 29 '22 09:09

Ernesto Campohermoso