How to send messages between c++ .dll and C# app using named pipe?

Question

I'm making an injected .dll written in C++, and I want to communicate with a C# app using named pipes.

Now, I am using the built in System.IO.Pipe .net classes in the C# app, and I'm using the regular functions in C++.

I don't have much experience in C++ (Read: This is my first C++ code..), tho I'm experienced in C#.

It seems that the connection with the server and the client is working, the only problem is the messaged aren't being send. I tried making the .dll the server, the C# app the server, making the pipe direction InOut (duplex) but none seems to work.

When I tried to make the .dll the server, which sends messages to the C# app, the code I used was like this:

DWORD ServerCreate() // function to create the server and wait till it successfully creates it to return.
{
    hPipe = CreateNamedPipe(pipename,//The unique pipe name. This string must have the following form:  \\.\pipe\pipename
    PIPE_ACCESS_DUPLEX, 
    PIPE_TYPE_MESSAGE | PIPE_READMODE_MESSAGE | PIPE_NOWAIT, //write and read and return right away
    PIPE_UNLIMITED_INSTANCES,//The maximum number of instances that can be created for this pipe
    4096 , // output time-out 
    4096 , // input time-out 
    0,//client time-out 
    NULL);

   if(hPipe== INVALID_HANDLE_VALUE)
   {
    return 1;//failed
   }
   else
    return 0;//success
}

void SendMsg(string msg)
{
    DWORD cbWritten;
    WriteFile(hPipe,msg.c_str(), msg.length()+1, &cbWritten,NULL);
}

void ProccesingPipeInstance()
{
    while(ServerCreate() == 1)//if failed
{
    Sleep(1000);    
}

//if created success, wait to connect
ConnectNamedPipe(hPipe, NULL);
for(;;)
{
    SendMsg("HI!");
    if( ConnectNamedPipe(hPipe, NULL)==0)
        if(GetLastError()==ERROR_NO_DATA)
        {
            DebugPrintA("previous closed,ERROR_NO_DATA");
            DisconnectNamedPipe(hPipe);
            ConnectNamedPipe(hPipe, NULL);
        }
    Sleep(1000);

}

And the C# cliend like this:

static void Main(string[] args)
    {
        Console.WriteLine("Hello!");

        using (var pipe = new NamedPipeClientStream(".", "HyprPipe", PipeDirection.In))
        {
            Console.WriteLine("Created Client!");

            Console.Write("Connecting to pipe server in the .dll ...");
            pipe.Connect();

            Console.WriteLine("DONE!");

            using (var pr = new StreamReader(pipe))
            {
                string t;
                while ((t = pr.ReadLine()) != null)
                {
                    Console.WriteLine("Message: {0}",t);
                }
            }
        }
    }

I see that the C# client connected to the .dll, but it won't receive any message.. I tried doing it the other way around, as I said before, and trying to make the C# send messages to the .dll, which would show them in a message box. The .dll was injected and connected to the C# server, but when It received a message it just crashed the application it was injected to.

Please help me, or guide me on how to use named pipes between C++ and C# app

Answer 1

A few things.

1- Are you using the same pipe name
C++ : "\\.\pipe\HyperPipe"
C# : "HyperPipe"

2- I think on the C# side it might be better to use ReadToEnd(), I have only used named pipes in C++, but I assume that ReadToEnd() will read a message since you are using message based pipes.

3- On the C++ side, you are trying to use non-blocking pipes and polling the pipe for connection and data etc. I would suggest one of three things.

  a - Use a blocking pipe on a separate thread or
  b - Use non blocking pipes using Overlapped IO
  c - Use non blocking pipes using IOCompletion ports

The first option would be the easiest and for what it sounds like you are doing, it will scale fine. Here is a link to a simple sample to (a) http://msdn.microsoft.com/en-us/library/aa365588(VS.85).aspx

4- Make sure that your encodings match on both sides. For example, if your C++ code is compiled for Unicode, you must read the Pipe stream on the C# side using Unicode encoding. Some thing like the following.

using (StreamReader rdr = new StreamReader(pipe, System.Text.Encoding.Unicode))
{
  System.Console.WriteLine(rdr.ReadToEnd());
}

Update: Since I had not worked with this in C# I thought I would write a small test. Just using blocking pipes no threading or anything, just to confirm the basics work, here is the very rough test code.

C++ Server

#include <tchar.h>
#include <windows.h>

int _tmain(int argc, _TCHAR* argv[])
{
  HANDLE hPipe = ::CreateNamedPipe(_T("\\\\.\\pipe\\HyperPipe"),
    PIPE_ACCESS_DUPLEX,
    PIPE_TYPE_MESSAGE | PIPE_READMODE_MESSAGE,
    PIPE_UNLIMITED_INSTANCES,
    4096,
    4096,
    0,
    NULL);


  ConnectNamedPipe(hPipe, NULL);

  LPTSTR data = _T("Hello");
  DWORD bytesWritten = 0;
  WriteFile(hPipe, data, _tcslen(data) * sizeof(TCHAR), &bytesWritten, NULL);
  CloseHandle(hPipe);
  return 0;
}

C# Client

using System;
using System.Text;
using System.IO;
using System.IO.Pipes;

namespace CSPipe
{
  class Program
  {
    static void Main(string[] args)
    {
      NamedPipeClientStream pipe = new NamedPipeClientStream(".", "HyperPipe", PipeDirection.InOut);
      pipe.Connect();
      using (StreamReader rdr = new StreamReader(pipe, Encoding.Unicode))
      {
        System.Console.WriteLine(rdr.ReadToEnd());
      }

      Console.ReadKey();
    }
  }
}