Serial Port not being flushed properly

Question

I have a RPi (Yes, I know, maybe someone thinks this belongs to the RPi site, but I think it's related to Linux in general, so StackOverflow is the right place) and I'm communicating between some Arduinos over a MAX485 using Python3 and pySerial. This works more or less, because I discovered that I need to do some weird workaround in order to get everything working "properly".

If I send data with:

GPIO.output(23, 1) # Pulling transmit pin high to send
comport.write("Some data".encode()) # Writing data
comport.flush() # Flushing the buffer
GPIO.output(23, 0) # Pulling pin down to receive

the Arduino receives the data and responds imediately, but because pySerial is somehow not ready yet, it becomes nothing back and there we have a lost packet.

However, if I try it this way:

GPIO.output(23, 1)
comport.write("Some data".encode())
time.sleep(.001) # Add some delay of only 1ms
comport.flush()
GPIO.output(23, 0)

Then the data gets send and also received. This lead me to a question: Is the flush command even working properly? I tried it out doing:

GPIO.output(23, 1)
comport.write("Some data".encode())
time.sleep(.001) # Add some delay
# -- No flush --
GPIO.output(23, 0)

And surprisingly it works too. It seems that the sleep "replaces" the flush command.

Why is the pySerial's buffer not flushing? I know, this might be a way to do it, but sleep only adds a (normally) unnecessary bit of code causing the whole code to wait (Over 500 lines) and this isn't that great.

I already searched around the internet and some said it is the USB TTL Adapter that doesn't support flushing (Which is not the case here, this one is onboard), others said it may be a Linux Kernel bug, so nothing really makes sense to me.

If someone could explain why the flush command isn't working and maybe how to get it working (if there's a way), I and probably every future visitant of this question would be very glad.

Answer 1

You were not specific but I think it is fair to assume that you are using the MAX485 in half-duplex mode. If this is the case, then what you are trying to do will not work reliably.

What is the problem?

Half Duplex communication uses the same pair of wires to transmit in either direction. But only one end can be transmitting at a time. Therefore this requires some sort of way to coordinate who is currently transmitting.

I beleive you have a good summary of the problem here:

the Arduino receives the data and responds imediately, but because pySerial is somehow not ready yet, it becomes nothing back and there we have a lost packet.

However it is not PySerial that is not ready, it is your MAX485, and the state of GPIO-23.

It is all in the timing

Here is a picture cribbed from an excellent tech note from Moxa on Half Duplex 485, titled The Secrets of RS-485 Half-duplex Communication

This picture shows that you will need to change the state of GPIO-23 (MASTER-RTS in this picture) at just the right moment, or your communication will fail. The tech note is worth reading in full and I believe well describes the challenges you are facing.

How do I fix it?

The answer is, it depends. Your problem could be switching GPIO-23 too early, or too late. If it is possible, the easiest, but least performant way is to get your Arduino to wait some time before responding. This will allow you sleep() for a bit to make sure your packet has finished transmitting, and then switch the state of GPIO-23 before the Arduino starts to respond.

If this solution is not possible or desirable, then you will need to investigate other ways to drive GPIO-23. There are many other ways to do this in SW, and some will work better than others. But note that this will never be 100% reliable using only python code to switch the GPIO-23 line. If high reliability is needed, HW assist of some sort is almost certainly necessary.

One final note, unless RS-485 is required, you could consider Full Duplex, using RS-422, or maybe even RS-232.

Answer 2

For pyserial you can set the write_timeout and timeout (for read) parameters wich would lead if not set to zero that the write and read operation will blocking.

So if you do not set the write_timeout parameter the default behavior of your write operation is to block until your data is send to the hardware. You do not need to flush the data, there will be nothing to do for this function.

Now it depends on how your configuration looks like and wich kernel driver you are using for your communication.

If you are using an embedded UART port of the RPi then the blocked write operation should wait until all data really pushed out. Otherwise this is probably an malconfigured kernel driver or a bug.

If you are using a USB to UART converter like an FTDI chip so the blocked wirte operation will return after the data are transferred to the chip. This means that the data maybe not fully transmitted by this ic when the write function returns.

How to handle this problem (you will need an oscilloscope): 1. Check the delay between asserting the tx pin and start of transmission. 2. Check the delay between deasserting the tx pin and end of transmission.

For small packets it will maybe help to increase the baud rate.

Furthermore you should considering using a hardware driven tx enable circuit. Several FTDI chips can be configured to behave like this (appropriate pin must be connected to tx enable).