Wrap native DLL for C#

Question

I wrote a C++ DLL and now I need to call a native function from a managed app.

The exported native function appears like this:

extern "C" __declspec(dllexport) 
bool NativeMethod(char *param1, char *param2, char *result);

So, from C# I'll call that function passing 2 input params, 1 output param and obviously I'll read the return bool value.

I tried to wrap all this in many ways, but always I get a PInvokeStackImbalance exception. The only way I know to call native function is by applying CallingConvention = CallingConvention.Cdecl) on .NET function declaration. However in this way I'm not able to read the output param (it's empty string always) and also the return value is always true.

Answer 1

You'll save yourself a lot of P/Invoke headaches if you just use COM Interop instead. Put the method in a COM interface and change the signature to follow COM conventions:

interface ISomeInterface : IUnknown
{
    HRESULT NativeMethod([in] BSTR bstrParam1, [in] BSTR bstrParam2, 
                         [out] BSTR* pbstrParam3, [out, retval] VARIANT_BOOL* pvbResult);
}

I changed char* to BSTR and bool to VARIANT_BOOL because those are the types used by COM for strings and bools, respectively. Also, all COM methods must return an HRESULT. If you want an "actual" return value you have to add it as the last out parameter and also mark it with the retval attribute.

Then add a reference to the COM component from the C# project and you'll get an intuitive C# signature without having to guess how to match C++ types with C# types:

bool NativeMethod(string bstrParam1, string bstrParam2, out string pbstrParam3)

(That's how it appears in Object Browser.)

Answer 2

why note using .Net code marshalling using DLLImport such as the following

[DllImport(@"C:\TestLib.dll")]
        public static extern void ProtectDocument(
            out [MarshalAs(UnmanagedType.LPStr)]string validToDate);

and then you can call the function as local function as the following

string x=string.empty;
ProtectDocument(out x); 

Answer 3

First, I'd adjust the prototype of your native function.

Since this function has a C interface, you should use a C type for booleans, not a C++ type like bool. You may want to use Win32's BOOL type.

Moreover, as it currently is, your function is prone to buffer overruns: it's better to add another parameter to specify the maximum size of the destination result string buffer.

Note also that a widespread calling convention for DLLs exporting pure C interface functions (like lots of Win32 API functions) is __stdcall (not __cdecl). I'd use that as well.

Last, since the first two parameters are input strings, you may want to use const to make it clear and enforce const-correctness.

So, I'd make the prototype of the exported native function like this:

extern "C" __declspec(dllexport) 
BOOL __stdcall NativeFunction(
    const char *in1, 
    const char *in2, 
    char *result, 
    int resultMaxSize);

Then, on the C# side, you can use the following P/Invoke:

   [DllImport(
        "NativeDll.dll", 
        CharSet = CharSet.Ansi, 
        CallingConvention = CallingConvention.StdCall)]
    [return: MarshalAs(UnmanagedType.Bool)]
    static extern bool NativeFunction(
        string in1,
        string in2,
        StringBuilder result, 
        int resultMaxSize);

Note that for the output string, a StringBuilder is used.

Note also that CharSet = CharSet.Ansi is used to marshal C#'s Unicode UTF-16 strings to ANSI (pay attention to the fact that the conversion is lossy - if you want a non-lossy conversion, just use wchar_t* strings on the C++ side as well).

I did a test with a simple C++ native DLL:

// NativeDll.cpp

#include <string.h>
#include <windows.h>

extern "C" __declspec(dllexport) 
BOOL __stdcall NativeFunction(
    const char *in1, 
    const char *in2, 
    char *result, 
    int resultMaxSize)
{
    // Parameter check
    if (in1 == nullptr 
        || in2 == nullptr 
        || result == nullptr 
        || resultMaxSize <= 0)
        return FALSE;

    // result = in1 + in2
    strcpy_s(result, resultMaxSize, in1);
    strcat_s(result, resultMaxSize, in2);

    // All right
    return TRUE;
}

And it is called successfully by the following C# console app code:

using System;
using System.Runtime.InteropServices;
using System.Text;

namespace CSharpClient
{
    class Program
    {
        [DllImport(
            "NativeDll.dll", 
            CharSet = CharSet.Ansi, 
            CallingConvention = CallingConvention.StdCall)]
        [return: MarshalAs(UnmanagedType.Bool)]
        static extern bool NativeFunction(
            string in1,
            string in2,
            StringBuilder result, 
            int resultMaxSize);

        static void Main(string[] args)
        {
            var result = new StringBuilder(200);
            if (! NativeFunction("Hello", " world!", result, result.Capacity))
            {
                Console.WriteLine("Error.");
                return;
            }

            Console.WriteLine(result.ToString());
        }
    }
}