How to use a Roslyn scripting submission as an assembly in other Roslyn compilations

Question

I'd like to reuse a script as a dynamic assembly in another non-scripting Roslyn compilation, but I can't for the life of me figure out how to make that work.

For example, say a I create a script the normal way and then emit the script as an assembly to a byte stream using something like:

var compilationOptions = new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary);
var compilation = script.GetCompilation().WithOptions(compilationOptions);
using (var ms = new MemoryStream())
{
    EmitResult result = compilation.Emit(ms);
    ms.Seek(0, SeekOrigin.Begin);
    assembly = Assembly.Load(ms.ToArray());
}

Now, let's say I want to feed that assembly into another non-scripting compilation as a reference. I can't just use the assembly since none of the MetadataReference.CreateFrom...() methods support passing an actual Assembly instance. As a dynamic assembly it doesn't have a location so I can't use MetadataReference.CreateFromFile().

In the past I've used MetadataReference.CreateFromStream() for this type of thing with success, but that doesn't seem to work when the assembly represents a script submission (I have no idea why). The compilation proceeds, but as soon as you attempt to use a type from the submission you get errors like:

System.InvalidCastException: [A]Foo cannot be cast to [B]Foo. Type A originates from 'R*19cecf20-a48e-4a31-9b65-4c0163eba857#1-0, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' in the context 'LoadNeither' in a byte array. Type B originates from 'R*19cecf20-a48e-4a31-9b65-4c0163eba857#1-0, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' in the context 'LoadNeither' in a byte array.

I'm guessing it has something to do with the submission assembly being in a different context when evaluating vs. loaded as a byte array. I'd love any insight or guidance into the best way to use objects and methods defined in a script submission in later non-script compilations.

Update 7/29

I was able to get a minimal repro that demonstrates the problem. It can be found at https://github.com/daveaglick/ScriptingAssemblyReuse.

In producing the repro, it became clear that an important component of this problem is that the script passes the Type of one of it's classes out to the calling code, the calling code then uses that Type to instantiate an instance of the object, and then passes the instance into the compilation that references the script assembly. The mismatch happens when casting from the instance of the type created by the host application to the type from within the referencing compilation. As I re-read that it sounds very confusing, so hopefully the code below will make it clearer.

Here's all the code to trigger this problem:

namespace ScriptingAssemblyReuse
{
    public class Globals
    {
        public IFactory Factory { get; set; }    
    }

    public interface IFactory
    {
        object Get();
    }

    public class Factory<T> : IFactory where T : new()
    {
        public object Get() => new T();
    }

    public class Program
    {
        public static void Main(string[] args)
        {
            new Program().Run();
        }

        private Assembly _scriptAssembly;

        public void Run()
        {
            AppDomain.CurrentDomain.AssemblyResolve += OnAssemblyResolve;

            // Create the script
            Script<object> script = CSharpScript.Create(@"
                public class Foo { }
                Factory = new ScriptingAssemblyReuse.Factory<Foo>();
                ", ScriptOptions.Default.WithReferences(MetadataReference.CreateFromFile(typeof(IFactory).Assembly.Location)), typeof(Globals));

            // Create a compilation and get the dynamic assembly
            CSharpCompilationOptions scriptCompilationOptions = new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary);
            Compilation scriptCompilation = script.GetCompilation().WithOptions(scriptCompilationOptions);
            byte[] scriptAssemblyBytes;
            using (MemoryStream ms = new MemoryStream())
            {
                EmitResult result = scriptCompilation.Emit(ms);
                ms.Seek(0, SeekOrigin.Begin);
                scriptAssemblyBytes = ms.ToArray();
            }
            _scriptAssembly = Assembly.Load(scriptAssemblyBytes);

            // Evaluate the script
            Globals globals = new Globals();
            script.RunAsync(globals).Wait();

            // Create the consuming compilation
            string assemblyName = Path.GetRandomFileName();
            CSharpParseOptions parseOptions = new CSharpParseOptions();
            SyntaxTree syntaxTree = CSharpSyntaxTree.ParseText(@"
                public class Bar
                {
                    public void Baz(object obj)
                    {
                        Script.Foo foo = (Script.Foo)obj;  // This is the line that triggers the exception 
                    }
                }", parseOptions, assemblyName);
            CSharpCompilationOptions compilationOptions = new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary);
            string assemblyPath = Path.GetDirectoryName(typeof(object).Assembly.Location);
            CSharpCompilation compilation = CSharpCompilation.Create(assemblyName, new[] {syntaxTree},
                new[]
                {
                    MetadataReference.CreateFromFile(Path.Combine(assemblyPath, "mscorlib.dll")),
                    MetadataReference.CreateFromFile(Path.Combine(assemblyPath, "System.dll")),
                    MetadataReference.CreateFromFile(Path.Combine(assemblyPath, "System.Core.dll")),
                    MetadataReference.CreateFromFile(Path.Combine(assemblyPath, "System.Runtime.dll"))
                }, compilationOptions);
            using (MemoryStream ms = new MemoryStream(scriptAssemblyBytes))
            {
                compilation = compilation.AddReferences(MetadataReference.CreateFromStream(ms));
            }

            // Get the consuming assembly
            Assembly assembly;
            using (MemoryStream ms = new MemoryStream())
            {
                EmitResult result = compilation.Emit(ms);
                ms.Seek(0, SeekOrigin.Begin);
                byte[] assemblyBytes = ms.ToArray();
                assembly = Assembly.Load(assemblyBytes);
            }

            // Call the consuming assembly
            Type barType = assembly.GetExportedTypes().First(t => t.Name.StartsWith("Bar", StringComparison.Ordinal));
            MethodInfo bazMethod = barType.GetMethod("Baz");
            object bar = Activator.CreateInstance(barType);
            object obj = globals.Factory.Get();
            bazMethod.Invoke(bar, new []{ obj });  // The exception bubbles up and gets thrown here
        }

        private Assembly OnAssemblyResolve(object sender, ResolveEventArgs args)
        {
            if (_scriptAssembly != null && args.Name == _scriptAssembly.FullName)
            {
                // Return the dynamically compiled script assembly if given it's name
                return _scriptAssembly;
            }
            return null;
        }
    }
}

Answer 1

I think I've solved the mystery. Here's what happens:

• Create a script.
• Get the script's compilation, but override the compilation options. Specifically, it uses the default ScriptClassName, which is Script rather than the one generated by the scripting API (e.g. Submission#0) - this is the crux of the matter.
• Emit an assembly using the overridden options & load it into memory from the stream.
• Run the script. At this point there are two distinct and incompatible assemblies with the same name in loaded into the AppDomain using byte arrays.
• Add the stream as a metadata reference to a new compilation and use it in code. It compiles fine since it's using the assembly generated from overridden options. You would not be able to compile it using the actual assembly created by the script, since names like Submission#0 are illegal in C#. If it weren't then you could put the actual script Assembly instance in the globals and use it in OnAssemblyResolve.
• Call method Baz with a parameter of type Submission#0+Foo and try to cast it to Script+Foo.

To conclude - I don't believe this is possible using the current Roslyn Scripting API. However, these APIs are not the only way to compile scripts. You can just create a compilation on your own and set the SourceCodeKind to Script. You'd have to do a lot yourself, like running the main script method, handling globals, etc. I've done something like this in RoslynPad because I wanted the script assemblies to load with PDBs (so exceptions would have line information).