ColdFusion 2016 performance versus ColdFusion 9

Question

We are in the process of upgrading from ColdFusion 9 to ColdFusion 2016 and we have noticed an overall decrease of performance. We ran several simulations to give more insight. Below is a script that gives a good example of the performance decrease. The script builds a query and then creates a structure from the query.

<!--- Machine info --->
<cfset runtime = createObject("java", "java.lang.System")>
<cfset props = runtime.getProperties()> 
<cfset env = runtime.getenv()> 

<Cfoutput>

coldfusion: #SERVER.ColdFusion.ProductVersion# #SERVER.ColdFusion.ProductLevel#<br>

java.version: #props["java.version"]#<br>
java.vm.name: #props["java.vm.name"]#<br>
os.name: #props["os.name"]#<br>

PROCESSOR_IDENTIFIER: #env["PROCESSOR_IDENTIFIER"]#<br>
PROCESSOR_ARCHITECTURE: #env["PROCESSOR_ARCHITECTURE"]#<br>
NUMBER_OF_PROCESSORS: #env["NUMBER_OF_PROCESSORS"]#<br><Br>
</Cfoutput>

<!--- Create a query --->
<cfset myQuery = QueryNew("Name, Time, Advanced", "VarChar, Time, Bit")> 
<cfset testQuery = QueryNew("ColumnA,ColumnB,ColumnC,ColumnD,ColumnE,ColumnF,ColumnG,ColumnH,ColumnI,ColumnJ,ColumnK,ColumnL,ColumnM,ColumnN","VarChar,VarChar,VarChar,VarChar,VarChar,VarChar,VarChar,VarChar,VarChar,VarChar,VarChar,VarChar,VarChar,VarChar")>

<!--- Populate the query --->
<Cfloop from=1 to=300 index="x">
    <cfset QueryAddRow(testQuery, 1)> 
    <cfloop index="intLetter" from="#Asc('A')#" to="#Asc('N')#" step="1">
        <cfset temp = QuerySetCell(testQuery, "Column#chr(intLetter)#", "Row #x# column #intLetter#", x)> 
    </cfloop>
</cfloop>

<Cfset init = GetTickCount()>
<!--- Query to structure --->
<Cfset queryToStruct = structNEw()>
<cfloop query="testQuery">
<Cfset init2 = GetTickCount()>
    <cfset queryToStruct[testQuery.currentrow] = structNew()>
    <cfset queryToStruct[testQuery.currentrow]['ColumnA'] = structNew()>
    <cfloop list="#testQuery.columnList#" index="key">
        <cfset queryToStruct[testQuery.currentrow]['ColumnA'][testQuery[key][testQuery.currentrow]] = testQuery[key][testQuery.currentrow]>
    </cfloop>
    <cfoutput>#x#:#GetTickCount()-init2#<br></cfoutput>
</cfloop>

<cfoutput>-----------<br><b>#GetTickCount()-init#</b><br><br><Br></cfoutput>

<!---Cfdump var=#queryToStruct# --->

We have two servers with the exact same hardware configuration. One server is running on Windows 2008 / ColdFusion Server 9 Enterprise (Java version 1.6.0_14) and the other one is running on Windows 2016 / ColdFusion 2016 Standard (Java version 1.8.0_112). Both ColdFusion servers have the same Minimum JVM Heap Size (5024 MB) and Maximum JVM Heap Size (5048 MB).

The performance on the ColdFusion 9 server is more then 4x faster. Can someone give an explanation of why this is happening and how to solve this?

Update

To rule out any other process that would slow down ColdFusion I installed ColdFusion 9, ColdFusion 11 and ColdFusion 2016 all on the same virtual machine and all using the built in web server. Default installation settings. The outcome: ColdFusion 9 is fastest, followed closely by ColdFusion 11. ColdFusion 2016 is much slower.

Update 2 Made some changes to the script, so it is more clear what this script is doing.

Update 3 The results can be viewed here: http://136.144.177.152/test2.asp or http://136.144.177.152/test-toma.asp or http://136.144.177.152/test-ag.asp Note that the code is actually processed, so each time you load the page the results differ slightly.

Also i would like to point out i am not trying to optimize this code. I tried to make a very simple reproducible example. The sole purpose is to point out the difference in performance and find a reason and a solution.

Update 4 Did some extra testing and found the potential problem. For some reason the following code is very slow on coldfusion 2016 / Windows 2016 :

<cfset tmp = testQuery['ColumnA'][testQuery.currentrow]>

What i found very strange is that updating the query value is not slow. E.g.

<cfset testQuery['ColumnA'][testQuery.currentrow] = key>

All results can be found here: http://136.144.177.152/test5.asp or http://136.144.177.152/test6.asp. I also installed coldfusion 2016 on my laptop and found no performance issues. I also tried installing coldfusion 2016 on windows 2012 machine. Here i found the same performance issues.

Update 5 Based on Tomalak suggestion I removed the indexed access notation. This clearly is a performance issue on coldfusion 2016. Actual results can be found here http://136.144.177.152/bug-adobe.asp. I opened a bug at adobe for this issue here https://tracker.adobe.com/#/view/CF-4201966.

Answer 1

I can't reproduce the problem, really. This runs quite fast on my ColdFusion 2016. I've created an optimized version of the QueryToStruct code, but other than that there's little difference.

I don't have a CF 9 server available, is this code also 4 times as fast on CF 9 when you test it?

<!--- Machine info --->
<cfset runtime = createObject("java", "java.lang.System")>
<cfset props = runtime.getProperties()>
<cfset env = runtime.getenv()>
<cfoutput>
ColdFusion: #SERVER.ColdFusion.ProductVersion# #SERVER.ColdFusion.ProductLevel#<br>
java.version: #props["java.version"]#<br>
java.vm.name: #props["java.vm.name"]#<br>
os.name: #props["os.name"]#<br>
PROCESSOR_IDENTIFIER: #env["PROCESSOR_IDENTIFIER"]#<br>
PROCESSOR_ARCHITECTURE: #env["PROCESSOR_ARCHITECTURE"]#<br>
NUMBER_OF_PROCESSORS: #env["NUMBER_OF_PROCESSORS"]#<br>
<br>
</cfoutput>

<!--- Create a query --->
<cfset ROWNUM = 2000>
<cfset testColumns = "ColumnA,ColumnB,ColumnC,ColumnD,ColumnE,ColumnF,ColumnG,ColumnH,ColumnI,ColumnJ,ColumnK,ColumnL,ColumnM,ColumnN">
<cfset testTypes = "VarChar,VarChar,VarChar,VarChar,VarChar,VarChar,VarChar ,VarChar,VarChar,VarChar,VarChar,VarChar,VarChar,VarChar">
<cfset testQuery = QueryNew(testColumns, testTypes)>

<!--- Populate the query --->
<cfloop from="1" to="#ROWNUM#" index="x">
    <cfset QueryAddRow(testQuery, 1)>
    <cfloop from="#Asc('A')#" to="#Asc('N')#" index="intLetter">
        <cfset QuerySetCell(testQuery, "Column#chr(intLetter)#", "#x#-#intLetter#", x)>
    </cfloop>
</cfloop>

<!--- Convert the query to a struct --->
<cfset init = GetTickCount()>
<cfset converted = QueryToStruct(testQuery, "ColumnA")>

<cfoutput>
<b>My version:</b> #StructCount(converted)# rows, #GetTickCount()-init# ms<br>
</cfoutput>

<!--- Convert the query to a struct --->
<cfset init = GetTickCount()>
<cfset converted = QueryToStructOP(testQuery)>

<cfoutput>
<b>OP version:</b> #StructCount(converted)# rows, #GetTickCount()-init# ms<br>
</cfoutput>

<!--- +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ --->
<cffunction name="QueryToStruct">
  <cfargument name="Query" type="query" required="yes">
  <cfargument name="IndexColumn" type="string" required="yes">

  <cfset var result = StructNew()>
  <cfset var rowTemplate = StructNew()>
  <cfset var key = "">
  <cfset var thisRow = "">

  <cfloop list="#Query.ColumnList#" index="key">
    <cfset rowTemplate[key] = "">
  </cfloop>

  <cfloop query="Query">
    <cfset thisRow = Duplicate(rowTemplate)>
    <cfset result[Query[IndexColumn][Query.CurrentRow]] = thisRow>
    <cfloop collection="#thisRow#" item="key">
      <cfset thisRow[key] = Query[key][Query.CurrentRow]>
    </cfloop>
  </cfloop>

  <cfreturn result>
</cffunction>

<cffunction name="QueryToStructOP">
  <cfargument name="Query" type="query" required="yes">

  <cfset var queryToStruct = StructNew()>
  <cfset var key = "">
  <cfset var index = "">

  <cfloop query="Query">
    <cfset index = Query['ColumnA'][Query.CurrentRow]>
    <cfset queryToStruct[index] = StructNew()>
    <cfloop list="#Query.ColumnList#" index="key">
      <cfset queryToStruct[index][Query[key][Query.CurrentRow]] = Query[key][Query.CurrentRow]>
    </cfloop>
  </cfloop>

  <cfreturn queryToStruct>
</cffunction>

Result (This is not even server hardware, and it's an old CPU on top of that):

java.version: 1.8.0_162
java.vm.name: Java HotSpot(TM) 64-Bit Server VM
os.name: Windows 7
ColdFusion: 2016,0,05,308055 Developer
PROCESSOR_IDENTIFIER: Intel64 Family 6 Model 42 Stepping 7, GenuineIntel
PROCESSOR_ARCHITECTURE: AMD64
NUMBER_OF_PROCESSORS: 4

My version: 2000 rows, 78 ms
OP version: 2000 rows, 229 ms

Answer 2

First, you're comparing CF 9 Enterprise to CF 2016 Standard. Shouldn't be a big deal for this example, but if you're regression testing your whole app on Standard, you'll see issues. My previous company migrated from 9 Ent to 2016 Ent and all we saw were performance improvements across the board. When you see bottlenecks, you should always consider refactoring. It's one reason why you upgrade.

The biggest problem is how you are converting a query to a struct. CF 2016 has much more advanced functionality. Compare your legacy process to this one that takes advantage of the member functions in query objects.

public array function arrayOfStructs(required query data) {
    var results = [];
    arguments.data.each(function(row) {
        arrayAppend(results, arguments.row);
    });
    return results;
}

The query member function each() references the contents of each row as a struct. No need to loop over each column, enter a new key and assign a value. Boom! Done. Fast as hell.

Do your upgrade. :)