C++ and C# Communication using Named Pipe

Question

I'm trying to reverse enginering a dll injected into a process, that does hook winsock send() and send data over a PipeStream.

This is the C# code that read the pipe stream:

    [StructLayout(LayoutKind.Sequential, Pack = 1, CharSet = CharSet.Ansi)]
    public struct PipeHeader
    {
        [MarshalAs(UnmanagedType.I1)]
        public byte command;
        [MarshalAs(UnmanagedType.I4)]
        public int sockid;
        public int datasize;
    }

    public static object RawDeserializeEx(byte[] rawdatas, Type anytype)
    {
        int rawsize = Marshal.SizeOf(anytype);
        if (rawsize > rawdatas.Length)
            return null;
        GCHandle handle = GCHandle.Alloc(rawdatas, GCHandleType.Pinned);
        IntPtr buffer = handle.AddrOfPinnedObject();
        object retobj = Marshal.PtrToStructure(buffer, anytype);
        handle.Free();
        return retobj;
    }
    private void PipeRead()
    {
        byte[] dbPipeMsgIn = new byte[9];
        byte[] zero = new byte[] { 0 };

        byte[] dbPipeMsgInData;
    PipeLoop:
        while (pipeIn.Read(dbPipeMsgIn, 0, 9) != 0)
        {
            strPipeMsgIn = (PipeHeader)RawDeserializeEx(dbPipeMsgIn, typeof(PipeHeader));
            if (strPipeMsgIn.datasize != 0)
            {
                dbPipeMsgInData = new byte[strPipeMsgIn.datasize];
                pipeIn.Read(dbPipeMsgInData, 0, dbPipeMsgInData.Length);
                //do something with dbPipeMsgInData
            }
        }
        if (pipeIn.IsConnected) goto PipeLoop;
    }

So far i have hooked the send() func, connect and send messages trough the pipe. The problem is that the data received is not the data i expect, so probably I do sent in a wrong way. I need help because i have very little to no C++ knowledge.

C++ code:

#pragma pack(1)
typedef struct
{
    byte command;
    int sockid;
    int datasize;
} PipeHeader;
#pragma pack(0)

int WINAPI MySend(SOCKET s, const char* buf, int len, int flags)
{
    PipeHeader ph;
    string p(buf);
    if(p.find("<TalkMsg") == 0){
        byte cmd = 0;
        ph.command = cmd;
        ph.sockid = s;
        ph.datasize = len;
        char buffer[sizeof(ph)];
        memcpy(buffer, &ph, sizeof(ph)); 
        if(SendPipeMessage(buffer, sizeof(buffer))){
            if(SendPipeMessage(buf, sizeof(buf))){
                MessageBox(NULL,"Message Sent", NULL, NULL);
            }
        }
        fopen_s(&pSendLogFile, "C:\\SendLog.txt", "a+");
        fprintf(pSendLogFile, "%s\n", buf);
        fclose(pSendLogFile);
    }
    return pSend(s, buf, len, flags);
}

BOOL SendPipeMessage(LPCVOID lpvMessage, DWORD ctToWrite){
    // Send a message to the pipe server. 
   cbToWrite = sizeof(lpvMessage);

   fSuccess = WriteFile( 
      hPipe,                  // pipe handle 
      lpvMessage,             // message 
      cbToWrite,              // message length 
      &cbWritten,             // bytes written 
      NULL);                  // not overlapped 

   if ( ! fSuccess) 
   {
      return false;
   }else return true;
}

Edit, more info:

The goal is to send a message containing a 9 Bytes long PipeHeader Struct over the pipe, then send another message containing the winsock send() data, (the buf variable), read the pipe on the C# application and parse the first message to get the DataSize of the next incoming message (that's the datasize var of PipeHeader), then, using the datasize read again the pipe to get the send() buffer. I think that's working in that way. I do not know very well how Pipes work.

Anyway the MAIN goal is to send the send() buffer from the C++ Dll into the C# application.

Update: Seems that i have to first serialize the PipeHeader struct in a way so i can Deserialize it using RawDeserializeEx() in the C# code. I tried by doing:

char buffer[sizeof(ph)];
memcpy(buffer, &ph, sizeof(ph)); 

The problem is that, in C++ sizeof(ph), or sizeof(buffer) return 12 bytes. Instead in C# the unmanaged size (Marshal.SizeOf()) of the same Struct, return 9 bytes.

Update 2: Solved the size differences by changing the struct packing. But I'm still not getting the right values in C#. Code Updated.

Answer 1

My first post, so go easy on me please :-)

BOOL SendPipeMessage(LPCVOID lpvMessage){
    // Send a message to the pipe server. 
   cbToWrite = sizeof(lpvMessage);

sizeof is used incorrectly. It will return size of type LPCVOID instead of size of buffer as you plan. You're sending 4 bytes (on x86), when you want to send 9 bytes.

In my opinion, you need to provide new parameter for your SendPipeMessage()

BOOL SendPipeMessage(LPCVOID lpvMessage, DWORD cbToWrite)

You can use sizeof in first call. On second call you may use len as argument.

if(SendPipeMessage((LPCVOID)&ph, sizeof(ph))){
    if(SendPipeMessage(buf, len)){

Answer 2

Didn't check the whole code as I'm out of time right now. But what I've noticed:

• I'm pretty sure you can't expect that an int is always 4 bytes in C++. Where size matters an int is not the best choice.
• You forgot a [MarshalAs(UnmanagedType.I4)] for the member datasize in your C# code. Attributes are only applied to the next item, not to all following items.
• Replace SendPipeMessage(buffer, sizeof(buffer)) with SendPipeMessage((char*)&ph, sizeof(ph)) to get the desired result. No need for memcpy to a char array there and as M.L. mentioned sizeof(buffer) is not what you want.
• Replace sizeof(buf) with len. You already have the length so why use sizeof? And sizeof(buf) is again not what you want.
• You can simplify your return code at the end to return fSuccess;. No need for if-else there. Or just return WriteFile(...);.
• Why you use cbToWrite = sizeof(lpvMessage);? This again is not what you want and you already pass ctToWrite which you should use instead in the call of WriteFile.