I have a serial device that I'm trying to read input from. I sent it a string "ID\r", and it returns "ID XX\r" (where \r is an ASCII carriage return, hex 0x0d).
Since the eol option on serial.readline is no longer supported, I'm using TextIOWrapper to read from the serial port and return a line at a time.
My problem is that instead of returning my string as soon as it sees the carriage return, it's waiting until the twice the timeout I set when I opened the serial port. I'd like it to return the string immediately as soon as it reads an entire line since I may have hundreds of these commands to send to the device and I don't want to wait for the timeout each time. If I set timeout to 0, then I get no output at all (presumably because my script stops waiting before the device has a chance to output anything), and if I set the timeout to None, the script blocks forever.
Here's a simple test script:
import serial
import io
import time
ser = serial.Serial("/dev/ttyUSB0", baudrate=9600,
bytesize=8, parity='N', stopbits=1,
xonxoff=0, rtscts=1, timeout=5)
sio = io.TextIOWrapper(io.BufferedRWPair(ser, ser),
newline=None)
sio.write(unicode("ID\r"))
sio.flush()
print "reading..."
x = sio.readline()
print len(x)
print x
The script always takes 10 seconds from the time it says "reading" until it prints the "ID XX" string that it read from the serial port.
I'm certain that the device is outputting the carriage return, as I've used strace to watch the reads:
select(4, [3], [], [], {5, 0}) = 1 (in [3], left {4, 991704})
read(3, "I", 8192) = 1
select(4, [3], [], [], {5, 0}) = 1 (in [3], left {4, 999267})
read(3, "D", 8191) = 1
select(4, [3], [], [], {5, 0}) = 1 (in [3], left {4, 999420})
read(3, " ", 8190) = 1
select(4, [3], [], [], {5, 0}) = 1 (in [3], left {4, 999321})
read(3, "X", 8189) = 1
select(4, [3], [], [], {5, 0}) = 1 (in [3], left {4, 999355})
read(3, "X", 8188) = 1
select(4, [3], [], [], {5, 0}) = 1 (in [3], left {4, 999171})
read(3, "\r", 8187) = 1
select(4, [3], [], [], {5, 0}) = 0 (Timeout)
select(4, [3], [], [], {5, 0}) = 0 (Timeout)
You can see the 2 select() timeouts that give the 10 second delay, but you can also clearly see the carriage return being read. I've tried setting the newline parameter to 'None' and '' (which should automatically allow \r, \n, and \r\n), and to '\r', but with the same result each time.
I've also tried setting the buffer_size in the BufferedRWPair() call to '1' to keep it from buffering input, but that made no difference.
Any idea what I'm doing wrong?
If I can't get this working, my next step will be to use serial.read() to read a character at a time and do my own line buffering, but I wanted to try to do it the "right" way with textiowrapper first.
The input buffer is computer memory allocated by the serial port object to store data that is to be read from the device. When reading data from your device, the data flow follows these two steps: The data read from the device is stored in the input buffer.
The RS232 Universal Serial Buffer can be used to buffer store serial data and includes a flash memory with a capacity of 4MB. The various operating modes provide great versatility.
Wasted a few hours on this today. It turned out that io.BufferedReader
reads until it has filled its buffer and then passes the buffer to io.TextIOWrapper
. The default buffer size is 8192, so depending on your device this might take a while.
The correct example code should be:
# buffer size is 1 byte, so directly passed to TextIOWrapper
sio = io.TextIOWrapper(io.BufferedRWPair(ser, ser, 1), encoding='ascii')
print sio.readline()[:-1]
Caveat: I am using Python 3.4 on a Mac so your mileage may vary, though I believe with TextIOWrapper backported to Python 2.7, the situation in Python 2.7 (and other OSs) will essentially be the same as what I describe below.
The main issue is that io.TextIOWrapper itself uses a buffering mechanism, controlled by the undocumented _CHUNK_SIZE attribute. This is pretty unpleasant. So you have two choices:
The second choice is to change _CHUNK_SIZE to 1. In your case, simply add the line
sio._CHUNK_SIZE = 1
to your code right after you initialized sio. This has the perhaps unpleasant effect that the buffering within TextIOWrapper itself will be turned off (this is used for the incremental decoding of the input). If performance is not an issue, this is the simplest solution. If performance is an issue, you can set a low value of timeout, not toucing _CHUNK_SIZE. However, in this case be prepared to get an empty string from readline() (if the device sends you an empty line, that will come through as '\n', so it can be distinguished from the empty string that you will get when a read runs out of the alloted time).
There is another problem with your code: When sio will be removed, the close method of ser will be called twice, which will result in an exception when your program will be about to finish (at least this is what happens if I try your code on my computer). You should create (it seems) to instances of serial.Serial and pass those to BufferedRWPair.
I have also created a wrapper class based on TextIOWrapper, which I could also post, if there is interest, just I did not want to litter response with some extra code which, strictly speaking, is not needed.
PS: In the meanwhile, I have experimented with the code on Ubuntu. While on my Mac, I did not see a need for setting the buffer size of io.BufferedRWPair to 1, on Ubuntu I had to do this, too, in addition to setting _CHUNK_SIZE to 1.
Thanks for the code Keith, but I wanted to keep this code somewhat portable, so I'd like to stick with the default "serial" package.
Plus, since I'm still learning Python, I wanted to try to learn how to use the TextIOWrapper in the way it was intended.
I gave up trying to make serial.readline() work, so for now I'll just use a simple "readLine" function to read a character at a time and look for a carriage return terminator. Though if I run into more serial quirkyness, I may revisit using your code.
Thanks!
def readLine(ser):
str = ""
while 1:
ch = ser.read()
if(ch == '\r' or ch == ''):
break
str += ch
#"print "str = " + str
return str
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With