.Net 4.0 app slower on 64 bit than 32 bit (Profiling and possible solutions) (app is using NetAdvantage)

Question

We have got .NET app written in VB .NET 4.0 / VS2010, compiled with all projects set to the AnyCPU setting for both Debug and Release configuration. We have noticed that when this app is run on a 64 bit environment (tested on Windows Server 2003 R2 and 2008 R2) that the app then takes at least double as long (in absolute terms about 25 secs) as opposed to around 6-12 seconds on a 32 bit environment (Win XP and 7) to start up.

I should add that the 64 bit systems are powerful servers, definitely more powerful than the other tested 32 bit systems. All other apps were faster on 64 bit, but not our poor app ;) (And we did test the apps at different times, under different load, and the results are always pretty much the same.)

As said above, the app is built using AnyCPU and it does run as 64 bit assembly under 64 bit OS (checked via TaskManager). The app itself is a WinForms app, making use of NetAdvantage Forms v10.3 and is regularly querying and writing to MS SQL Server 2008.

The different target machines are all on the same network, so the path to the database (same database was used for the performance tests) for example is the same, I don't think the problem is around the database or the network itself.

One thing I did notice, which seemed odd to me, was that when I built in different "profiling steps" using a stopwatch during the startup of our MainForm that the InitializeComponent method took twice as long on 64 bit, around 4 seconds opposed to 1.5 on 32 bit.

It's the very same app we deploy on both systems, no different config.

So I have got two questions:

Any idea what the cause of this could be?

And: What's the best way of determining the "offending" pieces of code? Currently I use stopwatches and try to narrow it down. But it looks to me like everything is slower on the 64 bit machines as far as our app is concerned, so I am not too sure I can break it down to specific statements.

Thanks all for your help, very much appreciated...

Answer 1

It turned out, that as soon as we switched from compiling from AnyCPU to specifically x86, i.e. also run as x86 on a x64 bit platform, we were back to our "good speed".

Answer 2

Had the same issue - and yes, the JIT is to blame. By judiciously using msgbox, narrowed it down to a method that would take 10 seconds to start (by calling message box just before calling the large method, and then as the first line of the large function.) And, yes, only saw this when compiling as AnyCPU, but not when explicitly x86; and not when running in debug.

In my case, it was a 5000 line Windows Forms InitializeComponent.

To prove this, run (elevated) "c:\windows\<.net framework dir>\ngen.exe install <myassembly.exe>" which will compile a native image. If this fixes it, then yes, the JIT was to blame.

Long term fixes, either:

• use ngen each time you deploy the program to rebuild the native image (or ngen update to rebuild, but obviously only works once installed once); (Downside is managing the ngen image(s) and the time it takes to ngen. This is the route I took, because there's an overall performance boost for larger applications.)

• or you can add the attribute <System.Runtime.CompilerServices.MethodImpl(MethodImplOptions.NoOptimization)> to the method. (Disables JIT on the method, so it's slower to execute but you don't pay the initial overhead of JITing, which is the the expensive part for us)

(I suspect that doing both is fruitless as that would mean giving up the native image of the large method for no gain.)