interop with nim return Struct Array containing a string /char* member

Question

interoping nim dll from c# i could call and execute the code below

if i will add another function (proc) that Calls GetPacks() and try to echo on each element's buffer i could see the output in the C# console correctly but i could not transfer the data as it is, i tried everything but i could not accomplish the task

proc GetPacksPtrNim(parSze: int, PackArrINOUT: var DataPackArr){.stdcall,exportc,dynlib.} =
  PackArrINOUT.newSeq(parSze)
  var dummyStr = "abcdefghij"
  for i, curDataPack in PackArrINOUT.mpairs:
    dummyStr[9] = char(i + int8'0')
    curDataPack = DataPack(buffer:dummyStr, intVal: uint32 i)

type
  DataPackArr = seq[DataPack]
  DataPack = object
    buffer: string
    intVal: uint32

when i do same in c/c++ the type i am using is either an IntPtr or char* that is happy to contain returned buffer member

EXPORT_API void __cdecl c_returnDataPack(unsigned int size, dataPack** DpArr)
{
    unsigned int dumln, Index;dataPack* CurDp = {NULL};
    char dummy[STRMAX];
    *DpArr = (dataPack*)malloc( size * sizeof( dataPack ));
    CurDp = *DpArr;
    strncpy(dummy, "abcdefgHij", STRMAX);

    dumln = sizeof(dummy);

    for ( Index = 0; Index < size; Index++,CurDp++)
    {
        CurDp->IVal = Index;
        dummy[dumln-1] = '0' + Index % (126 - '0');
        CurDp->Sval = (char*) calloc (dumln,sizeof(dummy));
        strcpy(CurDp->Sval, dummy);
    }

}

c# signature for c code above

    [DllImport(@"cdllI.dll", CallingConvention = CallingConvention.Cdecl), SuppressUnmanagedCodeSecurity]
    private static extern uint c_returnDataPack(uint x, DataPackg.TestC** tcdparr);

C# Struct

public unsafe static class DataPackg
{

   [StructLayout(LayoutKind.Sequential)]
    public struct TestC
    {
        public uint Id;
        public IntPtr StrVal;
    }

}

finally calling the function like so:

    public static unsafe List<DataPackg.TestC> PopulateLstPackC(int ArrL)
    {
        DataPackg.TestC* PackUArrOut;
        List<DataPackg.TestC> RtLstPackU = new List<DataPackg.TestC>(ArrL);
        c_returnDataPack((uint)ArrL, &PackUArrOut);
        DataPackg.TestC* CurrentPack = PackUArrOut;
        for (int i = 0; i < ArrL; i++, CurrentPack++)
        {

            RtLstPackU.Add(new DataPackg.TestC() { StrVal = CurrentPack->StrVal, Id = CurrentPack->Id });
        }
        //Console.WriteLine("Res={0}", Marshal.PtrToStringAnsi((IntPtr)RtLstPackU[1].StrVal));//new string(RtLstPackU[0].StrVal));
        return RtLstPackU;
    }

how could i produce similar c code as above from Nim ?

_{it doesn't have to be same code, but same effect, that in c# i would be able to read the content of the string. for now, the int is readable but the string is not}

Edit:

this is what i tried to make things simple struct array of int members

Update:

it seem that the problem is to do with my settings of nim in my windows OS. i will be updating as soon as i discover what exactly is wrong.

Answer 1

The string type in Nim is not equivalent to the C's const char* type. Strings in Nim are represented as pointers, pointing into a heap-allocated chunk of memory, which has the following layout:

NI length;   # the length of the stored string
NI capacity; # how much room do we have for growth
NIM_CHAR data[capacity]; # the actual string, zero-terminated

Please beware that these types are architecture specific and they are really an implementation detail of the compiler that can be changed in the future. NI is the architecture-default interger type and NIM_CHAR is usually equivalent to a 8-bit char, since Nim is leaning towards the use of UTF8.

With this in mind, you have several options:

1) You can teach C# about this layout and access the string buffers at their correct location (the above caveats apply). An example implementation of this approach can be found here: https://gist.github.com/zah/fe8f5956684abee6bec9

2) You can use a different type for the buffer field in your Nim code. Possible candidates are ptr char or the fixed size array[char]. The first one will require you to give up the automatic garbage collection and maintain a little bit of code for manual memory management. The second one will give up a little bit of space efficiency and it will put hard-limits on the size of these buffers.

EDIT: Using cstring may also look tempting, but it's ultimately dangerous. When you assign a regular string to a cstring, the result will be a normal char * value, pointing to the data buffer of the Nim string described above. Since the Nim garbage collector handles properly interior pointers to allocated values, this will be safe as long as the cstring value is placed in a traced location like the stack. But when you place it inside an object, the cstring won't be traced and nothing prevents the GC from releasing the memory, which may create a dangling pointer in your C# code.