How do I call a function in a C++ Dll from C# that has void* callback and object parameter

Question

I am trying to create a wrapper to a C dll and I am trying to call a function that has takes a callback function, receives an object as a pointer that is passed back.

The .h file delares

extern int SetErrorHandler(void (*handler) (int, const char*, void*),
                              void *data_ptr);

The handler is a callback function that is called when an error occurs and the data_ptr is any object (state) that is passed back to you, in the case of my app that will just be this (current object).

I am able to call functions in a dll that uses marshalled constant types like simple types strings, ints etc. But I cant figure out how to just marshall a pointer to a C# object that is the state.

In order to pass the object reference to the C function from what I have find by searching here and otherwise it seems that I need a structure type to be able to marshall to the function so I created a struct to hold my object:

[StructLayout(LayoutKind.Sequential)]
struct MyObjectState
{
   public object state;
}

EDIT: I tried to put an attribute: [MarshalAs(UnmanagedType.Struct, SizeConst = 4)] on the public object state property, but this produces the same error, so I removed it, doesnt seem it would work anyway.

The struct contains a single object property to hold any object for the callback function.

I declared the delegate in C# as follows:

delegate void ErrorHandler(int errorCode, IntPtr name, IntPtr data);

Then I declared the import function in C# as follows:

[DllImport("somedll.dll", CallingConvention = CallingConvention.Cdecl)]
static extern int SetErrorHandler handler, IntPtr data);

Then I created a callback function in my C# code:

void MyErrorHandler(int errorCode, IntPtr name, IntPtr data)
{
    var strName = Marshal.PtrToStringAnsi(name);
    var state = new MyObjectState();
    Marshal.PtrToStructure(data, state);
    Console.WriteLine(strName);
}

I am able to call the library function as follows:

var state = new MyObjectState()
{
    state = this
};
IntPtr pStruct = Marshal.AllocHGlobal(Marshal.SizeOf(state));
Marshal.StructureToPtr(state, pStruct, true);
int ret = SetErrorHandler(MyErrorHandler, pStruct);

The call works and the callback function is called but I am unable to access the data in the callback function and when i try Marshal.PtrToStructure I get an error:

The structure must not be a value class.

I did a lot of searching here and found various things on Marshall and void* but nothing has helped me to get this to work

Thanks.

Answer 1

You are making this more complicated than it needs to be. Your C# client does not need to use the data_ptr parameter because a C# delegate already has a built in mechanism for maintaining the this pointer.

So you can simply pass IntPtr.Zero to the delegate. Inside your error handler delegate you just ignore the value of data_ptr since this will be available.

In case you don't follow this description, here's a short program to illustrate what I mean. Note how MyErrorHandler is an instance method that acts as the error handler, and can modify instance data.

class Wrapper
{
    [UnmanagedFunctionPointer(CallingConvention.Cdecl)]
    delegate void ErrorHandler(int errorCode, string name, IntPtr data);

    [DllImport("somedll.dll", CallingConvention = CallingConvention.Cdecl)]
    static extern int SetErrorHandler(ErrorHandler handler, IntPtr data);

    void MyErrorHandler(int errorCode, string name, IntPtr data)
    {
        lastError = errorCode;
        lastErrorName = name;
    }

    public Wrapper()
    {
        SetErrorHandler(MyErrorHandler, IntPtr.Zero);
    }            

    public int lastError { get; set; }
    public string lastErrorName { get; set; }
}

class Program
{
    static void Main(string[] args)
    {
        Wrapper wrapper = new Wrapper();
    }
}

Answer 2

There may very well be a way to do this, but I gave up a long time ago. The solution I've come up with is slightly hackish, but it's very effective and works with everything I've thrown at it:

C# -> Managed C++ -> Native calls

Doing it this way you end up writing a small wrapper in managed C++, which is kind of a pain, but I found to be more capable and less painful than all of that marshaling code.

Honestly though I'm kind of hoping that someone gives a non-evasive answer, I've struggled with this for quite a while myself.