Writing unit tests in my compiler (which generates IL)

Question

I'm writing a Tiger compiler in C# and I'm going to translate the Tiger code into IL.

While implementing the semantic check of every node in my AST, I created lots of unit tests for this. That is pretty simple, because my CheckSemantic method looks like this:

public override void CheckSemantics(Scope scope, IList<Error> errors) {
...
}

so, if I want to write some unit test for the semantic check of some node, all I have to do is build an AST, and call that method. Then I can do something like:

Assert.That(errors.Count == 0);

or

Assert.That(errors.Count == 1);
Assert.That(errors[0] is UnexpectedTypeError);
Assert.That(scope.ExistsType("some_declared_type"));

but I'm starting the code generation in this moment, and I don't know what could be a good practice when writing unit tests for that phase.

I'm using the ILGenerator class. I've thought about the following:

1. Generate the code of the sample program I want to test
2. Save generated code as test.exe
3. Execute text.exe and store the output in results
4. Assert against results

but I'm wondering if there is a better way of doing it?

Answer 1

That's exactly what we do on the C# compiler team to test our IL generator.

We also run the generated executable through ILDASM and verify that the IL is produced as expected, and run it through PEVERIFY to ensure that we're generating verifiable code. (Except of course in those cases where we are deliberately generating unverifiable code.)

Answer 2

I've created a post-compiler in C# and I used this approach to test the mutated CIL:

1. Save the assembly in a temp file, that will be deleted after I'm done with it.
2. Use PEVerify to check the assembly; if there's a problem I copy it to a known place for further error analysis.
3. Test the assembly contents. In my case I'm mostly loading the assembly dynamically in a separate AppDomain (so I can tear it down later) and exercising a class in there (so it's like a self-checking assembly: here's a sample implementation).

I've also given some ideas on how to scale integration tests in this answer.