How to exchange binary data between processes in Linux

Question

I need to create an linux application that would do wireless network scanning, put the result in a structure and send it somehow to another, main application that will use the data. My initial idea was to create a pipe in the main application, fork and start another process by execl, that can write to the pipe. Something like this:

pid_t pid = NULL;
int pipefd[2];
FILE* output;
char line[256];

pipe(pipefd);
pid = fork();
if (pid == 0)
{
// Child
  close(pipefd[0]);
  dup2(pipefd[1], STDOUT_FILENO);
  dup2(pipefd[1], STDERR_FILENO);
  execl("/sbin/wifiscan", "/sbin/wifiscan", (char*) NULL);
}

//Only parent gets here. Listen to what the wifi scan says
close(pipefd[1]);
output = fdopen(pipefd[0], "r");

while(fgets(line, sizeof(line), output))
{
//Here we can listen to what wifiscan sends to its standard output
}

This however will not work with binary data, if binary 0 appears in the output. So I could either format the output of the wifiscan application to text, send it to pipe and parse in the main application, or do it in smarter way that I don't know yet.

What are other means to reliably exchange data between processes in Linux?

Answer 1

I suspect what's happening is that fgets() is reading the NUL characters properly, but you're interpreting the first one as being the end of the line that was read. It's tricky because fgets() is intended for textual input, and it does use the '\0' as a sentinel, not returning the number of characters read. Even if you know each line will be \n terminated, there's the posibility that raw binary data embedded in a line will include a \n too. So, switch to fread(), which is meant for binary. You can put a fixed-length (e.g. 2-byte, 4-byte) message size on the front of each message so the other side can read that first then do a fread() for the exact message size, avoiding messy issues with partial message reads.

If you really want to keep it some strange hybrid of text and binary, you could try using ftell() after fgets() to find out how much further along the stream you are, and hence how many characters should be in your buffer, but I've never seen a serious system do something so hacky.

For the record, without doing something markedly inefficient like hex-encoding every single character in the binary data, you could just encode the troublesome character(s). For example, the C string literal escape sequences could be used, with \0 representing a NUL and \\ a single \. Though less easy to visually check, it may be easier programmatically to encode/decode non-printable characters using say octal \NNN. If there are a lot of non-printable characters, then a base-64 uuencode approach - such as used in MIME email attachments - is another suitable but totally non-readable encoding.

Answer 2

There are many..

I would read Stevens.

http://www.kohala.com/start/unpv22e/unpv22e.html