Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

C# form from DLL loaded by native C++

Tags:

c++

c#

dll

winapi

This a question arising from this thread: Native C++ use C# dll via proxy C++ managed dll

In a nutshell, I'm loading (my) C# extension into a native process via a DLL. The extension needs to show a form so that the user can control it. I'm using standard .NET forms, no 3rd party librarys or anything, and my form is not showing up. Worse yet, it hangs the target process. It's not using any CPU, so I get the feeling its waiting for some function to return, but never does.

Also of interest is that the "Initialize method" message box pops up, but not the "Test" message box. I've tested everything I can think of (STAthread, threads, DisableThreadLibraryCalls, plus different code locations), every way to Sunday. I'm inclined to think it's some obscure detail of Win32 interop, but I can't find anything that would seem to cause these symptoms.

Can one of you experts take a look at my code and point out what the issue is?

/// <summary>
/// Provides entry points for native code
/// </summary>
internal static class UnmanagedExports
{
   [UnmanagedFunctionPointer(System.Runtime.InteropServices.CallingConvention.StdCall)]
   public delegate int SendRecv([MarshalAs(UnmanagedType.SafeArray)]byte[] ByteArray, UInt64 Len);

   [STAThread]
   [DllExport("Initialize", CallingConvention.StdCall)]
   public static int Initialize(IntPtr hInstance, SendRecv Send, SendRecv Recv)
   {
       return DLLinterface.Initialize(hInstance, Send, Recv);
   }

   [DllExport("Terminate", CallingConvention.StdCall)]
   public static void Terminate()
   {
       DLLinterface.Terminate();
   }
}

internal class DLLinterface
{
   static System.Threading.Thread uiThread;

   [STAThread]
   internal static int Initialize(IntPtr hInstance, UnmanagedExports.SendRecv Send, UnmanagedExports.SendRecv Recv)
   {
       MessageBox.Show("Initialize method");
       try
       {
           uiThread = new System.Threading.Thread(Run);
           uiThread.Start();
       }
       catch (Exception ex)
       {
           MessageBox.Show("Failed to load: " + ex.Message, "Infralissa error", MessageBoxButtons.OK, MessageBoxIcon.Error);
       }
       return 1;
   }

   [STAThread]
   private static void Run()
   {
       MessageBox.Show("Test");

       Application.EnableVisualStyles();
       Application.SetCompatibleTextRenderingDefault(false);
       Application.Run(new Form1());
   }

   internal static void Terminate()
   {
       MessageBox.Show("Terminating.");
       if (uiThread.IsAlive)
           uiThread.Abort();
   }
}
like image 749
Xcelled Avatar asked Nov 04 '22 20:11

Xcelled


1 Answers

It seems that the target itself was at fault. It wasn't loading the extensions directly, but instead loading a native "exensionManager.dll", where, as luck would have it, they were using DllMain to load up MY extension. In other words, I was trying to load a form under loaderlock and ran into a deadlock as. NET tried to load other assemblies.

The answer was simple, I had to show the form on a new thread. However, .NET's threading was also hanging, as it, too, required a library load which deadlocked.

In the end, I had to resort to a vanilla P/Invoke to CreateThread() directly, but the form is finally showing up.

like image 61
Xcelled Avatar answered Nov 09 '22 05:11

Xcelled