PInvokeStackImbalance exception when using IntPtr in .NET 4? (Works in .NET 3.5)

Question

Might be a bit of noob question but it's something that's been getting me in a pickle for the past few hours (or days)...

I'm calling a method from a DLL in my code in .NET Framework 4.0

  [DllImport("xeneth.dll")]
    public static extern ErrorCode XC_GetFrame(Int32 h, FrameType type, ulong ulFlags, IntPtr buff, uint size);

and then using it here:

if (XC_GetFrame(myCam, XC_GetFrameType(myCam), 0, IntPtr.Zero, (uint)fs) != ErrorCode.E_NO_FRAME)

However, when I run this in .NET 4.0 I get a P/INVOKE error, however... running this in 3.5 does not trigger this error. After myself and another programmer have gone over the code we seem to have put it down to the IntPtr running differently on 4.0.

My application needs to run in .NET 4.0 as a couple of features required by the application are only available in 4.0 ...

Is there anything that maybe i'm overlooking or have simply forgotten to include?

Any thoughts are much appreciated!

Tom

Update:

Native Declaration:

virtual ErrCode XCamera::GetFrame(FrameType type, unsigned long ulFlags, void *buffer, unsigned int size)

Error: A call to PInvoke function 'DLLTest!DLLTest.Form1::XC_GetFrameType' has unbalanced the stack. This is likely because the managed PInvoke signature does not match the unmanaged target signature. Check that the calling convention and parameters of the PInvoke signature match the target unmanaged signature.

Answer 1

Two common causes for this:

1. Mismatch of calling conventions. Your C# code uses the default of stdcall. Perhaps the native code uses cdecl. Try adding CallingConvention=CallingConvention.Cdecl to the DllImport attribute.
2. Mismatch of parameter lists (see below).

Don't kid yourself that your code is alright because .net 3.5 doesn't raise this error. The error detection in .net 4 is better which is why you only see the errors there. But your code is broken in all .net versions.

---

The native definition for your C++ virtual method is:

virtual ErrCode XCamera::GetFrame(FrameType type, unsigned long ulFlags, 
    void * buffer, unsigned int size);

It looks like you are passing the object pointer as the first parameter in your pinvoke call. I think that could work, although I don't know enough about how virtual functions are handled when exported to know whether that's a problem. Presumably you have exported a plain C function to instantiate objects.

The other problem that I see is that on Windows, C/C++ long is 32 bits. A C# long is 64 bits. This means that the correct declaration for ulFlags is as uint in your C# code.

Answer 2

Off the top of my head have you checked the bit-ness of both projects? You will get errors if one is 64-bit and the other is 32-bit. Note that in a recent version the default for C# projects changed from Any CPU to x86. That won't matter on a x86 OS, but on x64 OSes it makes a significant difference.

Additionally you should generally include as much of the text of the error as you can, censoring to protect private information is fine, but the error number and text can be quite helpful. Additionally when working with extern it is helpful to include both definitions of any struct types to ensure they are equivalent. (Another common problem).