How to open <del>named pipe</del>character device special file for reading and writing in Python

Question

I have a service running on a Linux box that creates a named pipe character device-special file, and I want to write a Python3 program that communicates with the service by writing text commands and reading text replies from the pipe device. I don't have source code for the service.

I can use os.open(named_pipe_pathname, os.O_RDWR), and I can use os.read(...) and os.write(...) to read and write it, but that's a pain because I have to write my own code to convert between bytes and strings, I have to write my own readline(...) function, etc.

I would much rather use a Python3 io object to read and write the pipe device, but every way I can think to create one returns the same error:

io.UnsupportedOperation: File or stream is not seekable.

For example, I get that message if I try open(pathname, "r+"), and I get that same message if I try fd=os.open(...) followed by os.fdopen(fd, "r+", ...).

Q: What is the preferred way for a Python3 program to write and read text to and from a named pipe character device?

---

Edit:

Oops! I assumed that I was dealing with a named pipe because documentation for the service describes it as a "pipe" and, because it doesn't appear in the file system until the user-mode service runs. But, the Linux file utility says it is in fact, a character device special file.

Answer 1

The problem occurs because attempting to use io.open in read-write mode implicitly tries to wrap the underlying file in io.BufferedRandom (which is then wrapped in io.TextIOWrapper if in text mode), which assumes the underlying file is not only read/write, but random access, and it takes liberties (seeking implicitly) based on this. There is a separate class, io.BufferedRWPair, intended for use with read/write pipes (the docstring specifically mentions it being used for sockets and two way pipes).

You can mimic the effects of io.open by manually wrapping layer by layer to produce the same end result. Specifically, for a text mode wrapper, you'd do something like:

rawf = io.FileIO(named_pipe_pathname, mode="rb+")
with io.TextIOWrapper(io.BufferedRWPair(rawf, rawf), encoding='utf-8', write_through=True) as txtf:
    del rawf   # Remove separate reference to rawf; txtf manages lifetime now
    # Example use that works (but is terrible form, since communicating with
    # oneself without threading, select module, etc., is highly likely to deadlock)
    # It works for this super-simple case; presumably you have some parallel real code
    txtf.write("abcé\n")
    txtf.flush()
    print(txtf.readline(), flush=True)

I believe this will close rawf twice when txtf is closed, but luckily, double-close is harmless here (the second close does nothing, realizing it's already closed).

Answer 2

Solution

You can use pexpect. Here is an example using two python modules:

caller.py

import pexpect

proc = pexpect.spawn('python3 backwards.py')
proc.expect(' > ')

while True:

    n = proc.sendline(input('Feed me - '))
    proc.expect(' > ')
    print(proc.before[n+1:].decode())

backwards.py

x = ''

while True:
    x = input(x[::-1] + ' > ')

Explanation

caller.py is using a "Pseudo-TTY device" to talk to backwards.py. We are providing input with sendline and capturing input with expect (and the before attribute).