How do I return a byte array from C++ to C#

Question

So I've been struggling with this for a little bit. I'm trying to make my own AES 128 library to use with one of my programs. The library tests out and works in C++ (well for the encrypt function.. I haven't implemented the others) The 'Encrypt' function is like this:

NEW CODE

void Aes128Class::EncryptBlock(BYTE* outBlock, const BYTE* inBlock, const BYTE* cipherBlock)
{
    BYTE temp[16] = {0x00};
    Galois::XorBlock(temp, inBlock);
    Galois::XorBlock(temp, cipherBlock);

    BYTE expandedKey[176] = {0x00};
    memcpy(expandedKey, Key, 16);
    Galois::expand_key(expandedKey);

    Galois::XorBlock(temp, expandedKey);
    for(int i=16; i<160; i+=16)
    {
        Galois::DoRound(temp, &expandedKey[i]);
    }
    Galois::SubBytes(temp);
    Galois::ShiftRows(temp);
    Galois::XorBlock(temp, &expandedKey[160]);

    memcpy(outBlock, temp, 16);
}
void Aes128Class::EncryptData(BYTE* outBlock, size_t& outlen, const BYTE* inBlock, size_t length)
{
    float blockSize = (float)(length/16);
    blockSize = ceilf(blockSize);
    int newLength = (int)(blockSize*16);
    BYTE* temp = (BYTE*)malloc(newLength);
    BYTE* padd = (BYTE*)malloc(newLength);
    memset(temp, 0, newLength);
    memcpy(padd, inBlock, length);
    EncryptBlock(temp, padd, IV);
    for (int i=1; i<blockSize; i++)
    {
        EncryptBlock(&temp[i*16], &padd[i*16], &temp[(i-1)*16]);
    }
    outlen = newLength;
    memcpy(outBlock, temp, newLength);
}

The idea is that if the plainText is not in a 16-byte block increment then I force it to be. So this makes for a variable size byte array. It works in my C++ tests, but when I call it in C# I get a few different errors... This will take a minute to describe.

    [DllImport("CppAes128.dll", CallingConvention = CallingConvention.ThisCall,
        EntryPoint = "?EncryptData@Aes128Class@@QAEXPAEAAIPBEI@Z")]
    static extern void EncryptData(IntPtr pClass, ref IntPtr outblock, [Out]int OutLength, byte[] inBlock, int length);

When I call this I get valid pointers to both the array, and the outlength. The way it looks right now causes an access violation, but I can get that structure to work if I change [Out]int OutLength to ref IntPtr. Interestingly, if I do ref int or ref uint it still "works". So if I do that I try to read the intptr and then I get an access violation. I am compiling this as a x86 project in .NET 4.0 (since I read somewhere that 3.5 had some bugs with access...)

Here is what I've tried in C#. It's a little garbled as I've been playing with it for hours (sorry):

    public byte[] EncryptData(byte[] plainText, int length)
    {
        byte[] enc = null;
        int len = 0;
        IntPtr pArray = IntPtr.Zero;
        EncryptData(theClass, ref pArray, len, plainText, length);

        Console.WriteLine(len);
        //enc = new byte[len];
        //Marshal.Copy(pArray, enc, 0, len);
        //Marshal.Release(pArray);
        //try
        //{
        //    int elementSize = Marshal.SizeOf(typeof(IntPtr));
        //    //IntPtr unmanagedArray = Marshal.AllocHGlobal(10 * elementSize);
        //    Console.WriteLine("Reading unmanaged memory:");
        //    // Print the 10 elements of the C-style unmanagedArray 
        //    for (int i = 0; i < 10; i++)
        //    {
        //        Console.WriteLine("{0:X2}:", Marshal.ReadByte(pArray, i));
        //    }

        //    Marshal.FreeHGlobal(pArray);

        //}
        //catch (Exception ex)
        //{
        //    Console.WriteLine("{0}\n{1}", ex.Source, ex.Message);
        //    Console.WriteLine("Win32({0})", Marshal.GetLastWin32Error());
        //}
        //Marshal.Release(pArray);
        return enc;
    }

The only time this worked is when I just made a static-size array and didn't use ref or marshal copy or anything.. I think my signature was something like this

static extern void EncryptData(IntPtr pClass, byte[] outBlock, byte[] inBlock, int length);

That almost worked, but the problem was that when I did a foreach loop on that array it was always the size that i put.. frustrating to say the least.

So what am I doing wrong? how can I get this to work? I am so frustrated with it. Thank you

Oh and FYI, this is so I cannot be dependent on the cryptlib anymore. I'm trying to recompile a different project, which uses cryptlib, as a static library and not shared, which causes some problems with my compiled options and is too big of a hassle to change back.

EDITED to show more code

This is the tests that I use. I found a webpage that showed a bunch of tests, so this is me implementing this.

void VerifyEncrypt16(const BYTE* expected, const BYTE* key, const BYTE* iv, const BYTE* plainText)
{
    BYTE actual[16] = {0x00};
    Aes128Class aes;
    aes.SetKey(key, 16);
    aes.SetIV(iv, 16);
    size_t len = 0;
    aes.EncryptData(actual, len, plainText, 16);
    _ASSERT(CompareTwoArrays(expected, actual));
}
void VerifyEncrypt16String(const char* expected, const char* key, const char* iv, const char* plainText)
{
    BYTE e[16];
    BYTE k[16];
    BYTE i[16];
    BYTE p[16];

    ByteUtil::StringToHex(expected, e);
    ByteUtil::StringToHex(key, k);
    ByteUtil::StringToHex(iv, i);
    ByteUtil::StringToHex(plainText, p);

    VerifyEncrypt16(e, k, i, p);
}
void CheckEncrypt16(void)
{
    _RPT0(_CRT_WARN, "Checking Encryption of a 16 byte number IV set to 0\n");
    //AESVS GFSbox test data for CBC
    VerifyEncrypt16String("0336763e966d92595a567cc9ce537f5e","00000000000000000000000000000000","00000000000000000000000000000000","f34481ec3cc627bacd5dc3fb08f273e6");
    VerifyEncrypt16String("a9a1631bf4996954ebc093957b234589","00000000000000000000000000000000","00000000000000000000000000000000","9798c4640bad75c7c3227db910174e72");
    VerifyEncrypt16String("ff4f8391a6a40ca5b25d23bedd44a597","00000000000000000000000000000000","00000000000000000000000000000000","96ab5c2ff612d9dfaae8c31f30c42168");
    VerifyEncrypt16String("dc43be40be0e53712f7e2bf5ca707209","00000000000000000000000000000000","00000000000000000000000000000000","6a118a874519e64e9963798a503f1d35");
    VerifyEncrypt16String("92beedab1895a94faa69b632e5cc47ce","00000000000000000000000000000000","00000000000000000000000000000000","cb9fceec81286ca3e989bd979b0cb284");
    VerifyEncrypt16String("459264f4798f6a78bacb89c15ed3d601","00000000000000000000000000000000","00000000000000000000000000000000","b26aeb1874e47ca8358ff22378f09144");
    VerifyEncrypt16String("08a4e2efec8a8e3312ca7460b9040bbf","00000000000000000000000000000000","00000000000000000000000000000000","58c8e00b2631686d54eab84b91f0aca1");

    //AESVS KeySbox test data for CBC
    VerifyEncrypt16String("6d251e6944b051e04eaa6fb4dbf78465","10a58869d74be5a374cf867cfb473859","00000000000000000000000000000000","00000000000000000000000000000000");
    //A TON OF MORE TESTS! etc etc etc        VerifyEncrypt16String("5c005e72c1418c44f569f2ea33ba54f3","00000000000000000000000000000000","00000000000000000000000000000000","fffffffffffffffffffffffffffffffe");
    VerifyEncrypt16String("3f5b8cc9ea855a0afa7347d23e8d664e","00000000000000000000000000000000","00000000000000000000000000000000","ffffffffffffffffffffffffffffffff");
}

Answer 1

In case you are still looking for answer, this example gives a starting point.

Basically, start from allocate memory block in native function call, than invoke call - back to managed where you pass ( by ref ) array and its size saved from the original list of input parameters.

This way you allocate memory block for managed code in managed code and edit it with content from native.

http://msdn.microsoft.com/en-us/library/ektebyzx.aspx

An alternate method would be nice to find, performance compare would be a bonus :)

Answer 2

Frankly, I've found the easiest way to do this is to wrap the unmanaged C++ call with a managed C++ call. In the managed C++, you can copy the data in a straight forward C++ manner (by pinning the data structure) and passing it back to C#