C# Performance MS verse Mono Problems

Question

I'm working on a fairly straight-forward (school) project. It's a job-shop scheduler. It's single-threaded, it has very limited File I/O (it reads a small problem description, then it goes to work trying to build a solution). The CPU should be the bottleneck. There is no user input/GUI.

On my machine, in release mode, without the debugger - in 3 minutes of CPU time, my PC can generate/evaluate 20,000 different schedules for a particular problem.

On a comparable *nix machine, executed with mono, in 3 minutes of CPU time, the server manages to generate/evaluate 2,000 different schedules. It's 1/10th the speed. I've compared Python performance between my machine and this particular server and the throughput was nearly identical.

The only 'system' call that I could see as being different was a call to

Process.GetCurrentProcess().TotalProcessorTime.Minutes

But removing it hasn't had any impact.

I've tried using

--aot -O=all

It didn't have any noticeable impact.

I've also tried to run the mono profiler against it but the results haven't been as helpful as I had hoped.

  Hits      % Method name
 57542  37.45 /usr/bin/mono
 11432   7.44 __lll_unlock_wake                    in /lib64/libpthread.so.0
  6898   4.49 System.Linq.Enumerable:Any<jobshop2.JobTask> (System.Collections.Generic.IEnumerable`1<jobshop2.JobTask>,System.Func`2<jobshop2.JobTask, bool>)
  6857   4.46 System.Collections.Generic.List`1/Enumerator<jobshop2.JobTask>:MoveNext ()
  3582   2.33 pthread_cond_wait@@GLIBC_2.3.2       in /lib64/libpthread.so.0
  2719   1.77 __lll_lock_wait                      in /lib64/libpthread.so.0

Of those top six lines - I only recognize two of them as being 'my code' that I could improve upon. In the full output I can see quite a few calls in /lib64/libpthread.so.0 that seem to deal with locking, unlocking, waiting, mutexes, and pthreads. I'm confused by this because it is not a multi-threaded application.

I'm going through the Performance page on the mono site but nothing is really jumping out at me as being a problem. I have no doubt that my code is ugly and slow, but I really wasn't expecting such a big performance drop. I'm currently trying to get Linux installed on my desktop so that I can run my app in mono on the same hardware to help eliminate that variable - but I thought someone might be able to offer some suggestions/insight.

EDIT: It is version 2.10.8 mono

Mono JIT compiler version 2.10.8 (tarball Sat Feb 16 11:51:56 UTC 2013)
Copyright (C) 2002-2011 Novell, Inc, Xamarin, Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       altstack
        Notifications: epoll
        Architecture:  amd64
        Disabled:      none
        Misc:          debugger softdebug
        LLVM:          supported, not enabled.
        GC:            Included Boehm (with typed GC and Parallel Mark)

Answer 1

This is a bit of an awkward answer, but I felt like it was the most fair way to handle it...I can't really explain what the cause was, but I don't want to imply that mono is horribly slow (it really isn't).

My concern was getting the program to run fast on the server. As other people have pointed out, the version of mono installed on the server was very old. I hope nobody sees my question and thinks that it reflects the current state of mono. Sadly, I am not able to update the version of mono on the server.

So, I re-wrote my code to remove unnecessary computation, avoid using iterators, and to limit memory allocations. My original code was doing a lot of unnecessary creation of objects and the objects were a lot larger than they needed to be. The clean up doubled the speed on my machine and made the 'server' performance about 70% of my own (a huge improvement!).

Still, it's not fair to compare different hardware - even if previous Python programs 'seemed' to run at about the same speed. I installed Linux and, with the newest version of mono, installed, my revised program ran at 96% of the Windows version.

I didn't keep digging beyond that. Using the current version of mono, on the same hardware, gave me nearly identical performance. Thanks for all the suggestions, it was incredibly helpful and saved me a lot of time.

Answer 2

Could be a memory leak. Mono is fighting an uphill battle; Microsoft made a system and developers had to reverse-engineer most of it. If you really can't figure it out, I would try reporting the bug to the mono developers:

Bugs - Mono (http://www.mono-project.com/Bugs)

Make sure that your mono version is up to date first; 2.10 is ancient. As of now, 3.2.6 is the latest. The packaged version from a package maintainer might not be good enough; try building it from the source tarball, and using that to run your program, before reporting bugs.

If you are using wine-mono or something like that on linux, then make sure that wine and wine-mono are up to date as well.