Avoid heap error while looping in coldfusion

Question

I am trying to loop over a 2-D array whose dimensions are 12000 * 20 and I am continuously getting java.lang.OutOfMemoryError.

Initially I thought this might be because of heap size so I increased my heap size but I was still getting the same error. So I ran a garbage collector like this:

<cflock name="checkMemory" type="exclusive" timeout="1" throwontimeout="yes">
    <cfset objSystem = CreateObject( "java", "java.lang.System" )>
    <cfset objSystem.gc()>
</cflock>

and I dumped the free memory which was around 850MB:

<cfset runtime = CreateObject("java","java.lang.Runtime").getRuntime()>
<cfset freeMemory = runtime.freeMemory()>
<cfdump var="#freeMemory#" label="free">

Here I am trying to create a XML variable and while looping I am getting heap error:

<cfxml variable="variables.XML">
  <cfoutput>
    <ROWS>
      <cfloop from="3" to="#arrayLen(local.array)#" index="i" step="1">
        <ROW>
          <cfloop from="1" to="#arrayLen(local.array[2])#" index="j" step="1">
            <#ucase(local.array[2][j])#>
              <![CDATA[#trim(local.array[i][j])#]]>
            </#ucase(local.array[2][j])#>
          </cfloop>
        </ROW>
      </cfloop>
    </ROWS>
  </cfoutput>
</cfxml> 

This is the stack trace:

java.lang.OutOfMemoryError at java.io.WinNTFileSystem.getBooleanAttributes(Native Method) at java.io.File.exists(File.java:733) at coldfusion.xml.XmlProcessor.getSourceURL(XmlProcessor.java:246) at coldfusion.xml.XmlProcessor.parse(XmlProcessor.java:155) at coldfusion.tagext.lang.XmlTag.doEndTag(XmlTag.java:85) at cffeeds2ecfc1003675922$funcDEMO1._factor8(C:\component\abc.cfc:1235) at cffeeds2ecfc1003675922$funcDEMO1.runFunction(C:\component\abc.cfc:1192) at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:472) at coldfusion.runtime.UDFMethod$ReturnTypeFilter.invoke(UDFMethod.java:405) at coldfusion.runtime.UDFMethod$ArgumentCollectionFilter.invoke(UDFMethod.java:368) at coldfusion.filter.FunctionAccessFilter.invoke(FunctionAccessFilter.java:55) at coldfusion.runtime.UDFMethod.runFilterChain(UDFMethod.java:321) at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:220) at coldfusion.runtime.CfJspPage._invokeUDF(CfJspPage.java:2582) at cffeeds2ecfc1003675922$funcDEMO.runFunction(\component\abc.cfc:935) at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:472) at coldfusion.runtime.UDFMethod$ReturnTypeFilter.invoke(UDFMethod.java:405) at coldfusion.runtime.UDFMethod$ArgumentCollectionFilter.invoke(UDFMethod.java:368) at coldfusion.filter.FunctionAccessFilter.invoke(FunctionAccessFilter.java:55) at coldfusion.runtime.UDFMethod.runFilterChain(UDFMethod.java:321) at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:517) at coldfusion.runtime.TemplateProxy.invoke(TemplateProxy.java:496) at coldfusion.runtime.TemplateProxy.invoke(TemplateProxy.java:355) at coldfusion.filter.ComponentFilter.invoke(ComponentFilter.java:188) at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:374) at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48) at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40) at coldfusion.filter.PathFilter.invoke(PathFilter.java:94) at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70) at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:28) at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38) at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46) at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38) at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22) at coldfusion.xml.rpc.CFCServlet.invoke(CFCServlet.java:139) at coldfusion.xml.rpc.CFCServlet.doPost(CFCServlet.java:290) at javax.servlet.http.HttpServlet.service(HttpServlet.java:760) at org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327) at javax.servlet.http.HttpServlet.service(HttpServlet.java:853) at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89) at jrun.servlet.FilterChain.doFilter(FilterChain.java:86) at com.intergral.fusionreactor.filter.FusionReactorFilter.i(FusionReactorFilter.java:566) at com.intergral.fusionreactor.filter.FusionReactorFilter.c(FusionReactorFilter.java:258) at com.intergral.fusionreactor.filter.FusionReactorFilter.doFilter(FusionReactorFilter.java:164) at jrun.servlet.FilterChain.doFilter(FilterChain.java:94) at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42) at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46) at jrun.servlet.FilterChain.doFilter(FilterChain.java:94) at jrun.servlet.FilterChain.service(FilterChain.java:101) at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106) at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42) at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286) at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543) at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203) at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320) at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428) at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266) at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

Is there any better solution to avoid looping like this or to resolve this error?

I created a cfm page locally and added some variables in it like this:

<cfset runtime = CreateObject("java","java.lang.Runtime").getRuntime()>
<cfset freeMemory = runtime.freeMemory()>
<cfset totalMemory = runtime.totalMemory()>
<cfset maxMemory = runtime.maxMemory()>
<cfdump var="#freeMemory#" label="free">
<cfdump var="#totalMemory#" label="total">
<cfdump var="#maxMemory#" label="max">

Every time I was refreshing this page the free memory size was decreasing until I ran the GC. I am still trying to figure out why this is happening. Any suggestions in this regard?

Please help. Thanks in advance.

Answer 1

After suggestions from Adam Cameron, Dan Bracuk and James Moberg, I have done the following and I am not getting the heap error now. I suspect that it was because of CFXML tag but not sure.

Converting all element of array[2] to uppercase:

<cfloop from="1" to="#arrayLen(local.array[2])#" index="i" step="1">
  <cfset local.array[2][i] = ucase(local.array[2][i])>
</cfloop>

Then using cfsavecontent to generate the xml string:

<cfsavecontent variable="local.xmlString">
  <?xml version="1.0" encoding="UTF-8"?>
  <ROWS>
    <cfoutput>
      <cfloop from="3" to="#arrayLen(local.array)#" index="local.currentRow" step="1">
        <ROW>
          <cfloop from="1" to="#arrayLen(local.array[2])#" index="local.currentColumn" step="1">
            <#local.array[2][local.currentColumn]#>
              <![CDATA[#trim(local.array[local.currentRow][local.currentColumn])#]]>
            </#local.array[2][local.currentColumn]#>
          </cfloop>
        </ROW>
      </cfloop>
    </cfoutput>
  </ROWS>
</cfsavecontent>
<cfset local.xml = xmlParse(local.xmlString)>