Communicating continuously between python script and C app

Question

I have a python script that interfaces with some network pool to read data from, that is continuously 320 bits. This 320 bits should be forward to some C app, that continuously reads these 320 bits from the python script and places them into an int array[8]. To be honest, I have no idea whether this is possible at all and would be thankful for a starting point for this issue.

I tried to incooperate some of your ideas, trying to send data via stdin from python to the C app:

test.exe:

#include <stdio.h>
int main(void)
{
    int ch;
    /* read character by character from stdin */
    do {
    ch = fgetc(stdin);
    putchar(ch);
    } while (ch != EOF);


    return 0;
}

test.py:

def run(self):

    while True:           
        payload = 'some data'
        sys.stdout.write(payload)
        time.sleep(5)

Then I start this whole thing using a pipe: python test.py | test.exe

Unfortunately, there is no data that is received on the test.exe side, shouldn't this data ba available on stdin?

Answer 1

There are several possible ways.

You could start the C program from the python program using the subprocess module. In that case you could write from the python program to the standard input of the C program. This is probably the easiest way.

import subprocess

network_data = 'data from the network goes here'
p = subprocess.Popen(['the_C_program', 'optional', 'arguments'], 
                     stdin=subprocess.PIPE)
p.stdin.write(network_data)

N.B. if you want to send data multiple times, then you should not use Popen.communicate().

Alternatively, you could use socket. But you would have to modifiy both programs to be able to do that.

Edit: J.F Sebatian's comment about using a pipe on the command line is very true, I forgot about that! But the abovementioned technique is still useful, because it is one less command to remember and especially if you want to have two-way communication between the python and C programs (in which case you have to add stdout=subprocess.PIPE to the subprocess.Popen invocation and then your python program can read from p.stdout).

Answer 2

Already mentioned:

1. sockets (be careful about framing)
2. message queues

New suggestions:

1. ctypes (package up the C code as a shared object and call into it from python with ctypes)
2. Cython/Pyrex (a Python-like language that allows you to pretty freely mix python and C data)
3. SWIG (an interlanguage interfacing system)

In truth, sockets and message queues are probably a safer way to go. But they likely will also mean more code changes.