pinvokestackimbalance -- how can I fix this or turn it off?

Question

I just switched to vs2010 from vs2008. Exact same solution, except now every single call to a C++ dll yields a 'pinvokestackimbalance' exception.

This exception does not get fired in 2008. I have complete access to the C++ dll and to the calling application. There does not appear to be any problem with the pinvoke, but this problem is making debugging other problems impossible; the IDE is stopping constantly to tell me about these things.

For instance, here's the C# signature:

    [DllImport("ImageOperations.dll")]
    static extern void FasterFunction(
        [MarshalAs(UnmanagedType.LPArray)]ushort[] inImage, //IntPtr inImage, 
        [MarshalAs(UnmanagedType.LPArray)]byte[] outImage, //IntPtr outImage, 
        int inTotalSize, int inWindow, int inLevel);

Here's what it looks like on the C++ side:

#ifdef OPERATIONS_EXPORTS
#define OPERATIONS_API __declspec(dllexport)
#else
#define OPERATIONS_API __declspec(dllimport)
#endif
extern "C" {


OPERATIONS_API void __cdecl FasterFunction(unsigned short* inArray, 
                                       unsigned char* outRemappedImage,
                                       int inTotalSize, 
                                       int inWindow, int inLevel);

}

What's different between vs2010 and vs2008 that would cause these exceptions to get thrown? Should I be adding a different set of parameters to the DllImport directive?

Answer 1

First, understand that the code is wrong (and always has been). The "pInvokeStackImbalance" is not an exception per se, but a managed debugging assistant. It was off by default in VS2008, but a lot of people did not turn it on, so it's on by default in VS2010. The MDA does not run in Release mode, so it won't trigger if you build for release.

In your case, the calling convention is incorrect. DllImport defaults to CallingConvention.WinApi, which is identical to CallingConvention.StdCall for x86 desktop code. It should be CallingConvention.Cdecl.

This can be done by editing the line [DllImport("ImageOperations.dll")] to be:

[DllImport("ImageOperations.dll", CallingConvention = CallingConvention.Cdecl)]

For more information, see this MSDN reference

Answer 2

To turn it off:

1. CTRL + ALT + E
2. Under "Managed Debugging Assistants" uncheck PInvokeStackImbalance.