A call to PInvoke function '[...]' has unbalanced the stack

Question

I'm getting this weird error on some stuff I've been using for quite a while. It may be a new thing in Visual Studio 2010 but I'm not sure.
I'm trying to call a unamanged function written in C++ from C#.
From what I've read on the internet and the error message itself it's got something to do with the fact that the signature in my C# file is not the same as the one from C++ but I really can't see it.
First of all this is my unamanged function below:

TEngine GCreateEngine(int width,int height,int depth,int deviceType);

And here is my function in C#:

[DllImport("Engine.dll", EntryPoint = "GCreateEngine", CallingConvention = CallingConvention.StdCall)]  
        public static extern IntPtr CreateEngine(int width,int height,int depth,int device);

When I debug into C++ I see all arguments just fine so thus I can only think it's got something to do with transforming from TEngine (which is a pointer to a class named CEngine) to IntPtr. I've used this before in VS2008 with no problem.

Answer 1

I had a _cdecl c++ dll that I called without any trouble from Visual Studio 2008, and then the identical code in Visual Studio 2010 would not work. I got the same PInvoke ... has unbalanced the stack error as well.

The solution for me was to specify the calling convention in the DllImport(...) attribute: From:

[DllImport(CudaLibDir)] 

To:

[DllImport(CudaLibDir, CallingConvention = CallingConvention.Cdecl)]

I guess they changed the default calling convention for DLLImport between .NET 3.5 and .NET 4.0?

Answer 2

It could also be that in the .NET Framework version 3.5, the pInvokeStackImbalance MDA is disabled by default. Under 4.0 (or maybe VS2010) it is enabled by default.

Yes. Technically, the code was always wrong, and previous versions of the framework silently corrected it.

To quote the .NET Framework 4 Migration Issues document: "To improve performance in interoperability with unmanaged code, incorrect calling conventions in a platform invoke now cause the application to fail. In previous versions, the marshaling layer resolved these errors up the stack... If you have binaries that cannot be updated, you can include the <NetFx40_PInvokeStackResilience> element in your application's configuration file to enable calling errors to be resolved up the stack as in earlier versions. However, this may affect the performance of your application."

An easy way to fix this is to specify the calling convention and make sure it is the same as in the DLL. A __declspec(dllexport) should yield a cdecl format.

[DllImport("foo.dll", CallingConvention = CallingConvention.Cdecl)]