Modbus Error: [Invalid Message] Incomplete message received, expected at least 2 bytes (0 received)

Question

Problem

pymodbus master/client can send a request to the slave/server. The slave/server make the things to return ready and is waiting for the master/client to pick them up. Despite the readiness of the server/slave, the master/client just returns the error "Modbus Error: [Input/Output] Modbus Error: [Invalid Message] Incomplete message received, expected at least 2 bytes (0 received)".

Setup

I use the laptop as server/slave with this adapter: https://www.amazon.com/dp/B076WVFXN8/ref=twister_B076X1BS4H?_encoding=UTF8&psc=1

I have an Raspberry Pi 3 / BananaPi as master/client with this adapter attached: https://www.aliexpress.com/item/32781613765.html?spm=a2g0s.9042311.0.0.1aec4c4d0EXx8M

I am following most of this tutorial for the setup, except for the Arduino is swapped with the laptop adapter: https://circuitdigest.com/microcontroller-projects/rs485-serial-communication-between-arduino-and-raspberry-pi - pin connections for the Raspberry is as in the tutorial.

I have this program as server/slave on my laptop:

#!/usr/bin/env python
from pymodbus.server.sync import StartTcpServer
from pymodbus.server.sync import StartUdpServer
from pymodbus.server.sync import StartSerialServer
from pymodbus.device import ModbusDeviceIdentification
from pymodbus.datastore import ModbusSequentialDataBlock, ModbusSparseDataBlock
from pymodbus.datastore import ModbusSlaveContext, ModbusServerContext
from pymodbus.transaction import ModbusRtuFramer, ModbusBinaryFramer

import logging
FORMAT = ('%(asctime)-15s %(threadName)-15s'
          ' %(levelname)-8s %(module)-15s:%(lineno)-8s %(message)s')
logging.basicConfig(format=FORMAT)
log = logging.getLogger()
log.setLevel(logging.DEBUG)

def run_server():

    slave_store1 = ModbusSlaveContext(co=ModbusSequentialDataBlock(0, [1]*16))
    slave_store2 = ModbusSlaveContext(di=ModbusSequentialDataBlock(0, [1]*16))
    slave_store3 = ModbusSlaveContext(ir=ModbusSequentialDataBlock(0, [5]*16))
    slave_store4 = ModbusSlaveContext(hr=ModbusSequentialDataBlock(0, [5]*16))

    slaves = {
        0x01: slave_store1,
        0x02: slave_store2,
        0x03: slave_store3,
        0x04: slave_store4,
    }

    context = ModbusServerContext(slaves=slaves, single=False)

    identity = ModbusDeviceIdentification()
    identity.VendorName = 'Pymodbus'
    identity.ProductCode = 'PM'
    identity.VendorUrl = 'http://github.com/riptideio/pymodbus/'
    identity.ProductName = 'Pymodbus Server'
    identity.ModelName = 'Pymodbus Server'
    identity.MajorMinorRevision = '2.2.0'

    # RTU:
    StartSerialServer(context, framer=ModbusRtuFramer, identity=identity, port='/dev/ttyUSB0', timeout=4, baudrate=115200, stopbits=1, bytesize=8, parity='N') 

if __name__ == "__main__":
    run_server()

The python version on server/slave is:

$ python3 --version
Python 3.5.2

And I start it with this command:

$ python3 pymodbus_sync_serv_example_2019.07.05-1316.py

I have the following as master/client on the Raspberry Pi 3 / BananaPi:

#!/usr/bin/env python

import logging
FORMAT = ('%(asctime)-15s %(threadName)-15s '
'%(levelname)-8s %(module)-15s:%(lineno)-8s %(message)s')
logging.basicConfig(format=FORMAT)
log = logging.getLogger()
log.setLevel(logging.DEBUG)

UNIT = 0x1

def run_sync_client():

    client = ModbusClient(method='rtu', port='/dev/ttyS2', timeout=4, baudrate=115200, stopbits=1, bytesize=8, parity='N')

    print(client)

    client.connect()

    log.debug("===================================")
    log.debug("Read input registers")
    log.debug("")
    rr = client.read_input_registers(1, 2, unit=3)
    print(rr)

    client.close()

if __name__ == "__main__":
    #for _ in range(10):
    run_sync_client()

Test and analyse

I have tried Raspberry Pi 3 as well as BananaPi. Same results.

I have tried baudrate= 9600, 38400 and now 115200.

timeout is already high as you can see in the code.

Logs for server/slave:

2019-07-07 13:35:00,333 MainThread      DEBUG    sync           :45       Client Connected [/dev/ttyUSB0:/dev/ttyUSB0]
2019-07-07 13:35:00,333 MainThread      DEBUG    sync           :522      Started thread to serve client
2019-07-07 13:35:08,341 MainThread      DEBUG    rtu_framer     :180      Getting Frame - 0x4 0x0 0x1 0x0 0x2
2019-07-07 13:35:08,341 MainThread      DEBUG    factory        :137      Factory Request[ReadInputRegistersRequest: 4]
2019-07-07 13:35:08,341 MainThread      DEBUG    rtu_framer     :115      Frame advanced, resetting header!!
2019-07-07 13:35:08,342 MainThread      DEBUG    context        :64       validate: fc-[4] address-2: count-2
2019-07-07 13:35:08,342 MainThread      DEBUG    context        :78       getValues fc-[4] address-2: count-2
2019-07-07 13:35:08,342 MainThread      DEBUG    sync           :143      send: [ReadRegisterResponse (2)]- b'030404000500050846'

The above server/slave just waits with a blinking curser after this last log line...

Logs for master/client:

ModbusSerialClient(rtu baud[115200])
2019-07-07 13:35:04,428 MainThread      DEBUG    pymodbus_sync_client_example_2019.07.05-1319:165      ===================================
2019-07-07 13:35:04,429 MainThread      DEBUG    pymodbus_sync_client_example_2019.07.05-1319:166      Read input registers
2019-07-07 13:35:04,430 MainThread      DEBUG    pymodbus_sync_client_example_2019.07.05-1319:167      
2019-07-07 13:35:04,430 MainThread      DEBUG    transaction    :111      Current transaction state - IDLE
2019-07-07 13:35:04,430 MainThread      DEBUG    transaction    :116      Running transaction 1
2019-07-07 13:35:04,431 MainThread      DEBUG    transaction    :215      SEND: 0x3 0x4 0x0 0x1 0x0 0x2 0x21 0xe9
2019-07-07 13:35:04,431 MainThread      DEBUG    sync           :73       New Transaction state 'SENDING'
2019-07-07 13:35:04,432 MainThread      DEBUG    transaction    :224      Changing transaction state from 'SENDING' to 'WAITING FOR REPLY'
2019-07-07 13:35:08,439 MainThread      DEBUG    transaction    :234      Transaction failed. (Modbus Error: [Invalid Message] Incomplete message received, expected at least 2 bytes (0 received)) 
2019-07-07 13:35:08,440 MainThread      DEBUG    rtu_framer     :235      Frame - [b''] not ready
2019-07-07 13:35:08,441 MainThread      DEBUG    transaction    :390      Getting transaction 3
2019-07-07 13:35:08,442 MainThread      DEBUG    transaction    :189      Changing transaction state from 'PROCESSING REPLY' to 'TRANSACTION_COMPLETE'
Modbus Error: [Input/Output] Modbus Error: [Invalid Message] Incomplete message received, expected at least 2 bytes (0 received)

The python version on master/client is:

$ python3 --version
Python 3.5.2

And I start it with this command:

$ python3 pymodbus_sync_client_example_2019.07.05-1319.py

The rights for the /dev's on the Raspberry/BananaPi are:

$ ls -l /dev/ttyS*
crw--w---- 1 root tty     249, 0 Jul  7 11:21 /dev/ttyS0
crw-rw---- 1 root dialout 249, 1 Jul  7 11:22 /dev/ttyS1
crw-rw---- 1 root dialout 249, 2 Jul  7 13:35 /dev/ttyS2
crw-rw---- 1 root dialout 249, 3 Jul  7 11:20 /dev/ttyS3

And on the server/slave on the laptop:

$ ls -l /dev/ttyUSB0
crw-rw---- 1 root dialout 188, 0 Jul  7 13:35 /dev/ttyUSB0

I have tried to send simple numbers with RS485 protocol. They can be send from master/Raspberry/BananaPi to the laptop, but not the other way around.

Have I the wrong rights settings for the devices?...

What am I doing wrong?...

What am I missing?...

As RS485 only works in the one way, I do not think that pymodbus is the problem (?)... (My logic says that pymodbus builds in the RS485 standard, and if that underlying layer of RS485 does not work, pymodbus will not. Is that assumption correct?)

I know some people are talking about that the Raspberry Pi is 3.3V on the pins and does not work with 5V pin-units. Despite that does all tutorials seem to ignore that fact and work. - Or are they just faking that it works? The TTL specifications say that all above 2.5V will be accepted as HIGH. SO in THEORY, 3.3V should be OK, just as the tutorials suggest.

I have by purpose yet not attached any resistors on the tx/rx wires for pull up/down. The tutorials don't suggest them.

I have tested the RS85 adapter sitting on the laptop with a modbus temperature-humidity sensor. This seems to work flawless. So this fact points in direction of BananaPi/Raspberry Pi and the RS485 adapter combination + software + settings to be flawed somehow.

Answer 1

First off, let me start saying it's a pleasure to answer such a well laid down question. Not everyone puts so much effort on explaining what they did and how they did it. Yours is a plus one question right after you finish reading it.

Now with your problem. You missed one very important step on the tutorial you followed. As you say Modbus is half-duplex¹, you have only two wires and only one device is allowed to talk on the bus so you need a way to take control of the bus, so to speak. In your USB-to-RS485/422 cable, that is done automatically for you by the hardware on the cable (your cable, in particular, uses the ubiquitous FTDI chip that has a TXEN -TX enable- signal, see here for more details), that's why you noticed the cable works well. On the other hand, your tiny 3$ transceiver is the poor brother, and it does not even have an UART, it's just a single-ended to differential converter. That's the reason you need to provide a DE/~RE (Drive Enable/Not Read Enable) signal for the poor guy to know when it is allowed to take control of the bus.

This is the warning you did not take from the tutorial:

IMPORTANT: Before writing values to the RS-485 module the pins DE & RE must be made HIGH.

That seems easy enough, but if you think how Modbus works... it's actually not so easy. This line of code:

rr = client.read_input_registers(1, 2, unit=3)

should be doing quite a number of things if you are to communicate with RS485 half-duplex successfully: take control of the bus (in your setup setting the RE/~DE signal high), send the Modbus query frame asking for two registers on UNIT ID 3, right after finishing writing the query (after 3.5 characters' time) release control of the bus (now setting RE/~DE low) and reading the answer from the slave.

As I explain in the link I already referred to above, there are several solutions to this problem. My preferred one (being more of a hardware guy) is doing the bus direction control signal by hardware (the best way is to have a transceiver that has this function implemented by hardware, like this one, but in the link you'll also find a DIY solution using a 555 timer). Now, if you prefer to do it the software way, you have some choices. You can tweak pymodbus to toggle the control line according to the Modbus needs (there are some links included in the answer I've quoted) or, if you prefer a more out-of-the-box solution use libmodbus.

If you decide for this last option, you can find all details on how to build and install lidmodbus with half-duplex support using the GPIO pins on the Rpi and if you want to stay on Python, install the wrapper and test the basic example. There are also a couple of scope screenshots to see the difference between toggling the line via software vs. hardware. For most in-house or hobbyist purposes, you should be able to use the software toggling but I would not trust it for industrial or more critical applications.

To finish off, I think it's worthwhile answering all your questions one by one:

As RS485 only works in the one way, I do not think that pymodbus is the problem (?)... (My logic says that pymodbus builds in the RS485 standard, and if that underlying layer of RS485 does not work, pymodbus will not. Is that assumption correct?)

Well, yes and no and maybe... As you read above, pymodbus is not really the problem. It is just expecting for you or your hardware to take care of the not so minor detail of controlling who accesses the bus. I think most people use this kind of library for Modbus TCP so this is never a problem for most users. In a general Modbus scenario where you have a PLC talking to another device via Modbus RTU on an RS485 link, the problem is dealt with by hardware, so you wouldn't have to worry about it either.

I know some people are talking about that the Raspberry Pi is 3.3V on the pins and does not work with 5V pin-units. Despite that does all tutorials seem to ignore that fact and work. - Or are they just faking that it works? The TTL specifications say that all above 2.5V will be accepted as HIGH. SO in THEORY, 3.3V should be OK, just as the tutorials suggest.

Correct, the MAX485 datahseet specifies the threshold values for VIH and VOL and as long as you use 5V for the power supply of your transceivers, the different logic levels won't be an issue (in this particular case, note that this is not a general statement, other devices might fail or end up destroyed if you mix logic levels).

I have by purpose yet not attached any resistors on the tx/rx wires for pull up/down. The tutorials don't suggest them.

Most likely you won't need to attach any terminating resistors to the bus for an in-house project. For long buses (in a factory or facility where devices can be hundreds of meters apart) you would probably worry about this issue. Your tiny transceiver actually has these terminating resistors already included so on its side, better not to add more resistance. For your cable, I did not have enough patience to find a manual (I don't know if there is one; I have a similar cable and the only way to be sure was to remove the cover and look under its hood).

Once you have everything up and running note that on your client:

print(rr)

Should be:

print(rr.registers)

If what you want is to show the values you have read.

Answer 2

Just as Marcos G. above suggest, did I modify pymodbus to control the GPIO chosen.

I chose the software solution as I just need some quick working stuff now without ordering new hardware and wait for that. I will later find a suitable/better hardware.

software solution that modifies pymodbus

Find the file "sync.py" in the folder "client", to modify the client/master side of your setup.

I modify the client/master here, as I have the 'poor' RS485 hardware on that side. If you have two of these 'poor' hardware things, you might need to modify the server side too.

The file sync.py can probably be found in

~/.local/lib/python3.5/site-packages/pymodbus/client

This may vary according to the python version you use. Mine is 3.5 for now. The "~/" part means that it is in your home folder. The dot in front of "local" makes the file hidden as standard. In a terminal can you use the command "ls -al" to show hidden files too. The graphical user interface of your Linux distribution will surely be able to show hidden files too somehow.

In the beginning of the file "sync.py", ad the following code:

import RPi.GPIO as GPIO
pin_de_re = 7
GPIO.setwarnings(False)
GPIO.setmode(GPIO.BOARD)
GPIO.setup(pin_de_re, GPIO.OUT, initial=GPIO.HIGH)

This can look somehow like the following:

more imports ...

from pymodbus.transaction import ModbusSocketFramer, ModbusBinaryFramer
from pymodbus.transaction import ModbusAsciiFramer, ModbusRtuFramer
from pymodbus.client.common import ModbusClientMixin

import RPi.GPIO as GPIO
pin_de_re = 7
GPIO.setwarnings(False)
GPIO.setmode(GPIO.BOARD)
GPIO.setup(pin_de_re, GPIO.OUT, initial=GPIO.HIGH)

# --------------------------------------------------------------------------- #
# Logging
# --------------------------------------------------------------------------- #
import logging
_logger = logging.getLogger(__name__)

...more code

Set the pin number as you choose. I have my control pin as GPIO4 - that is pin 7 in the Raspberry Pi/BananaPi.

Next you scroll down and find the section called

# --------------------------------------------------------------------------- #
# Modbus Serial Client Transport Implementation
# --------------------------------------------------------------------------- #

I modify this section, as I use Modbus RTU and hence serial for transmission of the data.

In that section you have to find the definition of "send":

    def _send(self, request):
        """ Sends data on the underlying socket

Inside that function, find the line:

            size = self.socket.write(request)

And embrace it with the control of the pin:

            _logger.debug("GPIO - Setting pin high")
            GPIO.output(pin_de_re, 1)
            time.sleep(.300)
            size = self.socket.write(request)
            time.sleep(.300)
            _logger.debug("GPIO - Setting pin low")
            GPIO.output(pin_de_re, 0)

The reason I use the lines '_logger.debug("GPIO - Setting pin high/low")' is that I then can see in the log in the terminal, that the program executes these things, and I can be assured if they are performed. If they don't show up in the log, I have done it in the false place - or something else...

The reason to use time.sleep(.300) is to let the hardware have time for act. .300 is 0.3 seconds. A big number in this context.

When I use the above solution, I get the following logs.

Slave/server:

2019-07-07 23:08:43,532 MainThread      DEBUG    sync           :45       Client Connected [/dev/ttyUSB0:/dev/ttyUSB0]
2019-07-07 23:08:43,533 MainThread      DEBUG    sync           :522      Started thread to serve client
2019-07-07 23:08:47,534 MainThread      DEBUG    rtu_framer     :232      Frame check failed, ignoring!!
2019-07-07 23:08:47,535 MainThread      DEBUG    rtu_framer     :128      Resetting frame - Current Frame in buffer - 0x3 0x4 0x0 0x1 0x0 0x82
2019-07-07 23:08:59,543 MainThread      DEBUG    rtu_framer     :180      Getting Frame - 0x4 0x0 0x1 0x0 0x2
2019-07-07 23:08:59,544 MainThread      DEBUG    factory        :137      Factory Request[ReadInputRegistersRequest: 4]
2019-07-07 23:08:59,544 MainThread      DEBUG    rtu_framer     :115      Frame advanced, resetting header!!
2019-07-07 23:08:59,544 MainThread      DEBUG    context        :64       validate: fc-[4] address-2: count-2
2019-07-07 23:08:59,544 MainThread      DEBUG    context        :78       getValues fc-[4] address-2: count-2
2019-07-07 23:08:59,545 MainThread      DEBUG    sync           :143      send: [ReadRegisterResponse (2)]- b'030404000500050846'

Master/client:

ModbusSerialClient(rtu baud[115200])
2019-07-07 23:08:55,839 MainThread      DEBUG    pymodbus_sync_client_example_2019.07.05-1319:165      ===================================
2019-07-07 23:08:55,840 MainThread      DEBUG    pymodbus_sync_client_example_2019.07.05-1319:166      Read input registers
2019-07-07 23:08:55,841 MainThread      DEBUG    pymodbus_sync_client_example_2019.07.05-1319:167      
2019-07-07 23:08:55,842 MainThread      DEBUG    transaction    :111      Current transaction state - IDLE
2019-07-07 23:08:55,842 MainThread      DEBUG    transaction    :116      Running transaction 1
2019-07-07 23:08:55,843 MainThread      DEBUG    transaction    :215      SEND: 0x3 0x4 0x0 0x1 0x0 0x2 0x21 0xe9
2019-07-07 23:08:55,843 MainThread      DEBUG    sync           :79       New Transaction state 'SENDING'
2019-07-07 23:08:55,844 MainThread      DEBUG    sync           :538      GPIO - Setting pin high
2019-07-07 23:08:55,845 MainThread      DEBUG    sync           :541      GPIO - Setting pin low
2019-07-07 23:08:55,845 MainThread      DEBUG    transaction    :224      Changing transaction state from 'SENDING' to 'WAITING FOR REPLY'
2019-07-07 23:08:59,516 MainThread      DEBUG    transaction    :300      Changing transaction state from 'WAITING FOR REPLY' to 'PROCESSING REPLY'
2019-07-07 23:08:59,518 MainThread      DEBUG    transaction    :229      RECV: 0x3 0x4 0x4 0x0 0x5 0x0 0x5 0x8 0x46
2019-07-07 23:08:59,519 MainThread      DEBUG    rtu_framer     :180      Getting Frame - 0x4 0x4 0x0 0x5 0x0 0x5
2019-07-07 23:08:59,519 MainThread      DEBUG    factory        :266      Factory Response[ReadInputRegistersResponse: 4]
2019-07-07 23:08:59,520 MainThread      DEBUG    rtu_framer     :115      Frame advanced, resetting header!!
2019-07-07 23:08:59,521 MainThread      DEBUG    transaction    :379      Adding transaction 3
2019-07-07 23:08:59,522 MainThread      DEBUG    transaction    :390      Getting transaction 3
2019-07-07 23:08:59,522 MainThread      DEBUG    transaction    :189      Changing transaction state from 'PROCESSING REPLY' to 'TRANSACTION_COMPLETE'
ReadRegisterResponse (2)

The transmission does MAYBE not always, but it pinpoints the cause of the problem and hence potential solutions.

I do not yet know what I will end up with. More stable hardware is for sure.

Regards modifying pymodbus or other software with respect to this issue, would I like to quote from the following post in another threat:

Anyone running modbus on a multitasking OS such as linux or windows will never be able to meet the requirements of the serial spec, there is no debate on this, tasking is normally 10ms so meeting 3.5us timing requirement just doesn't fit and never will.

A solution on the hardware side is preferable.

Thanks to Marcos G.