Delphi DLL return string from C# ... .NET 4.5 Heap Corruption but .NET 4.0 works? Explain please?

Question

I've been learning about marshaling Unmanaged DLL imports into C# ... And I've come across something I don't quite understand.

In Delphi, there is a function that is returning Result := NewStr(PChar(somestring)) from a Procedure SomeFunc() : PChar; Stdcall;

From my understanding, NewStr just allocates a buffer on the local heap ... and SomeFunc is returning a pointer to it.

In .NET 4.0 (Client Profile), via C# I can use :

[DllImport("SomeDelphi.dll", EntryPoint = "SomeFunc", CallingConvention = CallingConvention.StdCall)]
public static extern String SomeFunc(uint ObjID);

This works (or as David says, "appears to work") fine in Windows 7 .NET 4.0 Client Profile. In Windows 8, it has unpredictable behavior, which brought me down this path.

So I decided to try the same code in .NET 4.5 and got Heap corruption errors. Okay, so now I know this is not the correct way to do things. So I dig further :

Still in .NET 4.5

[DllImport("SomeDelphi.dll", EntryPoint = "SomeFunc", CallingConvention = CallingConvention.StdCall)]
public static extern IntPtr _SomeFunc();
public static String SomeFunc()
{
    IntPtr pstr = _SomeFunc();
    return Marshal.PtrToStringAnsi(pstr);
}

This works without a hitch. My (novice) concern is that NewStr() has allocated this memory and it's just sitting there forever. Is my concern not valid?

In .NET 4.0, I can even do this and it never throws an exception :

[DllImport("SomeDelphi.dll", EntryPoint = "SomeFunc", CallingConvention = CallingConvention.StdCall)]
public static extern IntPtr _SomeFunc();
public static String SomeFunc()
{
    String str;
    IntPtr pstr = _SomeFunc();
    str = Marshal.PtrToStringAnsi(pstr);
    Marshal.FreeCoTaskMem(pstr);
    return str;
}

This code throws the same heap exception in 4.5, however. This leads me to believe the problem lies in the fact that in .Net 4.5, the marshaler is trying to FreeCoTaskMem() and that is what is throwing the exceptions.

So questions :

1. Why does this work in .Net 4.0 and not 4.5?

2. Should I be concerned about NewStr()'s allocation in the native DLL?

3. If answer "No" to #2, then the second code example is valid?

Answer 1

The key information, which is very hard to find in the docs, concerns what the marshaller does with a p/invoke function with a return value of type string. The return value is mapped to a null-terminated character array, i.e. LPCTSTR in Win32 terms. So far so good.

But the marshaller also knows that the string must have been allocated on a heap somewhere. And it cannot expect the native code to deallocate it since the native function has finished. So the marshaller deallocates it. And it also assumes that the shared heap that was used was the COM heap. So the marshaller calls CoTaskMemFree on the pointer returned by the native code. And that's what leads to your error.

The conclusion is that if you wish to use string return value on the C# p/invoke end, you need to match that on the native end. To do so return PAnsiChar or PWideChar and allocate the character arrays with a call to CoTaskMemAlloc.

You absolutely cannot use NewStr here. In fact you should never call that function. Your existing code is comprehensively broken and every call you make to NewStr leads to a memory leak.

Some simple example code that will work:

Delphi

function SomeFunc: PAnsiChar; stdcall;
var
  SomeString: AnsiString;
  ByteCount: Integer;
begin
  SomeString := ...
  ByteCount := (Length(SomeString)+1)*SizeOf(SomeString[1]);
  Result := CoTaskMemAlloc(ByteCount);
  Move(PAnsiChar(SomeString)^, Result^, ByteCount);
end;

C#

[DllImport("SomeDelphi.dll")]
public static extern string SomeFunc();

You would probably want to wrap the native code up in a helper for convenience.

function COMHeapAllocatedString(const s: AnsiString): PAnsiChar; stdcall;
var
  ByteCount: Integer;
begin
  ByteCount := (Length(s)+1)*SizeOf(s[1]);
  Result := CoTaskMemAlloc(ByteCount);
  Move(PAnsiChar(s)^, Result^, ByteCount);
end;

Yet another option is to return a BSTR and use MarshalAs(UnmanagedType.BStr) on the C# side. However, before you do so, read this: Why can a WideString not be used as a function return value for interop?

---

Why do you see different behaviour in different .net versions? Hard to say for sure. Your code is just as broken in both. Perhaps the newer versions are better at detecting such errors. Perhaps there's some other difference. Are you running both 4.0 and 4.5 on the same machine, same OS. Perhaps your 4.0 test is running on an older OS which doesn't throw errors for COM heap corruptions.

My opinion is that there's little point understanding why broken code appears to work. The code is broken. Fix it, and move on.

Answer 2

My few points:

1. First, Marshal.FreeCoTaskMem is for freeing COM allocated memory blocks! It's not guaranteed to work for other memory blocks allocated by Delphi.

2. NewStr is deprecated (I get this after googling):

   NewStr(const S: string): PString; deprecated;

My suggestion is that you also export a DLL function that does string deallocation instead of using FreeCoTaskMem.