Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Unmanaged callback causing stack to overflow

I am working with an unmanaged resource with C#. The resource exposes a callback that can be setup for certain events that could happen within the hardware. To gain access to the unmanaged functions I do the following:

[DllImportAttribute("testDLL.dll", EntryPoint = "InstallCallback")]
public static extern short InstallCallback(uint handle, byte x, byte y, IntFuncPtr ptr);

[UnmanagedFunctionPointer(CallingConvention.StdCall)]
public delegate void IntFuncPtr(uint handle, byte x, byte y, LogEntry info);

I first install the callback with a reference to a method that follows the IntFuncPtr delegate. I then let the hardware go do its thing. After roughly 4700 calls of the callback the application crashes. The callback works fine if I write the code in c/c++ however I can replicate it by removing the __stdcall from my callback function. From C# I cannot catch the error which indicates the application is dying in the unmanaged resource. With the c/c++ application I can see that the stack overflows without __stdcall.

I thought that the delegate might not work with the calling convention stdcall so I tried the following:

[DllImportAttribute("testDLL.dll", EntryPoint = "InstallCallback")]
public static extern short InstallCallback(uint handle, byte x, byte y, IntPtr ptr);

public delegate void IntFuncPtr(uint handle, byte x, byte y, LogEntry info);

var callBackDelegate = new IntFuncPtr(Callback);
var callBackPtr = Marshal.GetFunctionPointerForDelegate(callBackDelegate);
InstallCallback(handle, 1, 1, callBackPtr);

This also did not work.

To summarize, I have an unmanaged callback that requires a function pointer to a function defined as a __stdcall. If the function pointer is to a non __stdcall function then the stack grows and overflows. I am attempting to use the callback in C# using DllImport and an UnmanagedFunctionPointer delegate with stdcall calling convention. When I do this the C# application acts like a c/c++ application that uses a non __stdcall function.

How can I get this to work entirely in C#?

Edit 1:

Here is the native method definition & structure information including the C# structure information.

extern "C" __declspec( dllexport ) short __stdcall InstallCallback(unsigned int handle, unsigned char x, unsigned char y, LOG_ENTRY info );

typedef union
{
    unsigned int ul_All;
    struct
    {
    unsigned int ul_Info:24;
    unsigned int uc_IntType:8;
    }t;

    struct
    {
    unsigned int ul_Info:24;

    unsigned int uc_Biu1:1;
    unsigned int uc_Biu2:1;
    unsigned int uc_Dma:1;
    unsigned int uc_Target:1;
    unsigned int uc_Cmd:1;
    unsigned int uc_Biu3:1;
    unsigned int uc_Biu4:1;
    unsigned int res:1;
    }b;
} LOG_ENTRY_C;

typedef union
{
    unsigned int All;
    struct
    {
    AiUInt32 Index:16;
    AiUInt32 Res:8;
    AiUInt32 IntSrc:8;
    }t;
} LOG_ENTRY_D;

typedef struct log_entry
{
    unsigned int a;
    unsigned int b;
    LOG_ENTRY_C c;
    LOG_ENTRY_D d;
} LOG_ENTRY;

[StructLayoutAttribute(LayoutKind.Sequential)]
public struct LogEntry {
    public uint Lla;
    public uint Llb;
    public LogEntryC Llc;
    public LogEntryD Lld;
}

[StructLayoutAttribute(LayoutKind.Explicit)]
public struct LogEntryC {
    [FieldOffsetAttribute(0)]
    public uint All;
    [FieldOffsetAttribute(0)]
    public LogEntryCT t;
    [FieldOffsetAttribute(0)]
    public LogEntryCB b;
}

[StructLayoutAttribute(LayoutKind.Explicit)]
public struct LogEntryD {
    [FieldOffsetAttribute(0)]
    public uint All;
    [FieldOffsetAttribute(0)]
    public LogEntryDT t;
}

[StructLayoutAttribute(LayoutKind.Sequential)]
public struct LogEntryCT {
    public uint bitvector1;
    public uint IntType {
        get { return ((uint)((this.bitvector1 & 255u))); }
        set { this.bitvector1 = ((uint)((value | this.bitvector1))); }
    }
    public uint Info {
        get { return ((uint)(((this.bitvector1 & 4294967040u) / 256))); }
        set { this.bitvector1 = ((uint)(((value * 256) | this.bitvector1))); }
    }
}
[StructLayoutAttribute(LayoutKind.Sequential)]
public struct LogEntryCB {
    public uint bitvector1;
    public uint res {
        get { return ((uint)((this.bitvector1 & 1u))); }
        set { this.bitvector1 = ((uint)((value | this.bitvector1))); }
    }
    public uint Biu4 {
        get { return ((uint)(((this.bitvector1 & 2u) / 2))); }
        set { this.bitvector1 = ((uint)(((value * 2) | this.bitvector1))); }
    }
    public uint Biu3 {
        get { return ((uint)(((this.bitvector1 & 4u) / 4))); }
        set { this.bitvector1 = ((uint)(((value * 4) | this.bitvector1))); }
    }
    public uint Cmd {
        get { return ((uint)(((this.bitvector1 & 8u) / 8))); }
        set { this.bitvector1 = ((uint)(((value * 8) | this.bitvector1))); }
    }
    public uint Target {
        get { return ((uint)(((this.bitvector1 & 16u) / 16))); }
        set { this.bitvector1 = ((uint)(((value * 16) | this.bitvector1))); }
    }
    public uint Dma {
        get { return ((uint)(((this.bitvector1 & 32u) / 32))); }
        set { this.bitvector1 = ((uint)(((value * 32) | this.bitvector1))); }
    }
    public uint Biu2 {
        get { return ((uint)(((this.bitvector1 & 64u) / 64))); }
        set { this.bitvector1 = ((uint)(((value * 64) | this.bitvector1))); }
    }
    public uint Biu1 {
        get { return ((uint)(((this.bitvector1 & 128u) / 128))); }
        set { this.bitvector1 = ((uint)(((value * 128) | this.bitvector1))); }
    }
    public uint Info {
        get { return ((uint)(((this.bitvector1 & 4294967040u) / 256))); }
        set { this.bitvector1 = ((uint)(((value * 256) | this.bitvector1))); }
    }
}

[StructLayoutAttribute(LayoutKind.Sequential)]
public struct LogEntryDT {
    public uint bitvector1;
    public uint IntSrc {
        get { return ((uint)((this.bitvector1 & 255u))); }
        set { this.bitvector1 = ((uint)((value | this.bitvector1))); }
    }
    public uint Res {
        get { return ((uint)(((this.bitvector1 & 65280u) / 256))); }
        set { this.bitvector1 = ((uint)(((value * 256) | this.bitvector1))); }
    }
    public uint Index {
        get { return ((uint)(((this.bitvector1 & 4294901760u) / 65536))); }
        set { this.bitvector1 = ((uint)(((value * 65536) | this.bitvector1))); }
    }
}
like image 319
Lux782 Avatar asked Nov 14 '22 13:11

Lux782


1 Answers

Seems to be a memory leak problem. Do you know if you need to free any memory related to the received objects (like the LogEntry)?

I have a similar scenario where I need to free the memory of every object passed to my callback method.

Review your C# code and try to identify what you are doing different from the c/c++ one.

like image 125
Carlos Bomtempo Avatar answered Nov 16 '22 02:11

Carlos Bomtempo