how could someone make a c# incremental compiler like Java?

Question

Years ago someone asked why c# doesn't allow incremental compilation like Java. El Skeet said it is to do with Java outputting .class files rather than assemblies.

Now that its 2011 and groovy things like the Mono compiler-as-a-service have been released, what would need to be done to make an incremental compiler for c#?

edit: to everyone banging on about how this isn't a problem, here's a quote from Jon Skeet from the thread I linked to :

Are you suggesting you never find yourself waiting for a build? Even 15 seconds? If a build takes 15 seconds and you want to build 20 times in an hour (which I certainly do with TDD) that means I'm wasting 5 minutes. Taking a 5 minute break is one thing - that's a good way of relaxing etc - but being held up for 15 seconds 20 times can be very frustrating. It's not long enough to do anything useful (other than maybe sip a drink) but it's long enough to irritate.

I suspect two factors contribute the level of annoyance I feel which others apparently don't: 1) TDD really relies on a faster turnaround 2) When working with Java in Eclipse, such delays are very rare

Answer 1

If it was not done then there is only one reason for it: efforts to do it are higher than possible benefits.

Microsoft will definitely not do it because costs are too high: .net code lives in assemblies and no one will change it. And yes, assemblies prevent class-by-class incremental compilation. No one will stop using assemblies.

And here is my answer why no one needs it. You can distribute your classes that constitute single project among several assemblies and compile them one by one. It is actually incremental compilation but not as fine-grained as class-by-class incremental compilation. And when your architecture is properly designed assembly level incremental compilation is sufficient.

Edit: Okay, I downloaded Mono C# compiler to take a look it is possible to make it incremental. I think it is not very hard. Basically it does following steps: 1) Parse files 2) Compile 3) Create assembly. You could hook somewhere after types are compiled and save then into some sort of intermediate files. Then recompile only changed ones. So it is possible, but looks like it is not high-priority issue for Mono team.

Edit 2: I found this interesting thread where people discuss Incremental compilation for Mono C# compiler. It is rather old but key explanation might be still valid:

Lexing and parsing normally are very fast and depend only on the size of the code being parsed. Semantic analysis is normally the most time consuming step as loading referenced assemblies and sifting around the huge metadata to resolve symbols and types is really the meat of the compiler, also, new "compiled" code is "appended" to this metadata/AST what increases the complexity of resolving symbols over time. Emission of code is done in memory first so it is fast. Saving to disk is slow but depends on emitted code size.

For incremental compiling, caching the metadata, would make everything very fast, as normally very little would be changed from one compilation to the other. But gmcs would have to invalidate only part of the metadata/AST, what it wasn't built for.

Edit 3: C# compiler had /incremental option in v1.0 and v1.1, but it was removed:

The /incremental flag found in the 1.0 and 1.1 version of the C# compiler is now considered obsolete.

Edit 4: Miguel de Icaza gives clear answer (1, 2) why Mono Compiler will not be incremental:

There are many, many more places where GMCS was just not designed to work on an edit-and-continue scenario.

If someone wants to make this their thesis subject, that is fine with me, but the amount of changes are too large in too many areas. I do not even want to bother enumerating them.

The reason I did not list things is because they will be everywhere in the compiler. Am sure you will run into them as soon as you try them out ;-)

So he considers it to be a task huger than for one man's thesis. And Mono has much more outstanding and actual tasks.