EOFError: Ran out of input inside a class

Question

I have the following code where I need to read multiple sensors at a time. I have set up threading and multiprocessing to do this task for me. When the threading and mutliprocessing code is outside of the main class, it works fine but the class can't use the data it retrives. When I put the mutlithreading code insdie the class, I run into an EOFError: Ran out of input error.

Here is the code:

import os
import multiprocessing
from multiprocessing import Process, Pool
import threading
import queue
import tkinter as tk
from tkinter import *
from tkinter import ttk
import time
import minimalmodbus
import serial
minimalmodbus.CLOSE_PORT_AFTER_EACH_CALL = True
THREAD_LOCK = threading.Lock()

class Application(tk.Frame):
    def __init__(self, master=None):
        super().__init__(master)

        self.pack()

        self.first_gas_labelframe = LabelFrame(self, text="Gas 1", width=100)
        self.first_gas_labelframe.grid(row=0, column=0)

        self.value_label = Label(self.first_gas_labelframe, text="Value")
        self.value_label.grid(row=0, column=0)

        self.unit_label = Label(self.first_gas_labelframe, text="Unit")
        self.unit_label.grid(row=1, column=0)

        self.temp_label = Label(self.first_gas_labelframe, text="Temp")
        self.temp_label.grid(row=2, column=0)

        self.temp_label6 = Label(self.first_gas_labelframe6, text="Temp")
        self.temp_label6.grid(row=2, column=0)

        self.timer_button = tk.Button(self, text='Start', command=self.start_run)
        self.timer_button.grid(row=2, column=0)


    def start_run(self):
        self.all_thread()

    def all_thread(self):
        thread = threading.Thread(target=self.all_process)
        thread.start()

    def all_process(self):
        all_ports = port_num()
        gas = minimalmodbus.Instrument("COM3", 1)
        gas.serial.baudrate = 9600
        gas.serial.bytesize = 8
        gas.serial.parity = serial.PARITY_NONE
        gas.serial.stopbits = 1
        gas.serial.timeout = 0.25
        gas.mode = minimalmodbus.MODE_RTU

        gas_list = [gas]
        processes = []
        while len(gas_list) > 0:
            val = 1
            with THREAD_LOCK:
                for sen in gas_list:
                    proc = Process(target=self.main_reader, args=(sen, val))
                    processes.append(proc)
                    proc.start()
                    val += 1
                for sen in processes:
                    sen.join()
                time.sleep(1)

    def main_reader(sen, val):
        try:
            read = sen.read_registers(0,42)
        except OSError:
            read = "Communication Error"
        except ValueError:
            read = "RTU Error"
        print(read)

if __name__ == '__main__':
    root = tk.Tk()
    root.geometry("1000x600")
    app = Application()
    app.mainloop()

With some debugging, the problem happens at proc.start() but proc has data. The lists have data too, which is why I am confused why it is running out of input. Note: in my code there are six entries in the gas_list

Answer 1

You cannot use multiprocessing like that (well, you can, but the result will be unpredictable) - when you create a new process your minimalmodbus.Instrument object from the list doesn't get passed as a reference but as a whole new object. Python essentially runs a completely new Python interpreter instance whenever you start() a multiprocess.Process instance and since different processes get different stacks they don't get to share the internal memory so Python actually pickles the passed arguments, sends them to the Process and then unpickles them there creating an illusion that both processes (the parent and the child) have the same data.

You can observe it yourself if instead of creating a new multiprocessing.Process you call self.main_reader(pickle.loads(pickle.dumps(sen)), val) (val also gets pickled but as a generic it's not of any importance here).

The very same process happens to the Application.main_reader() method (although weirdly defined), too - the way you have it set up is that your whole Application instance actually gets recreated in the sub-process so that Python can call its main_reader() method.

What you can do instead is to pass needed arguments to recreate your original object to the sub-process function, and then have your object created when your function starts. For example, if you modify your Application.all_process() method as:

def all_process(self):
    gas = {"__init__": ("COM3", 1)
           "serial": {
               "baudrate": 9600,
               "bytesize": 8,
               "parity": serial.PARITY_NONE,
               "stopbits": 1,
               "timeout": 0.25
           },
           "mode": minimalmodbus.MODE_RTU}

    gas_list = [gas]
    processes = []
    while len(gas_list) > 0:
        val = 1
        for sen in gas_list:
            # we'll be calling the main_reader function outside of the Application instead
            proc = multiprocessing.Process(target=main_reader, args=(sen, val))
            processes.append(proc)
            proc.start()
            val += 1
        for sen in processes:
            sen.join()
        time.sleep(1)
        # you do plan to exit this loop, right?

And then have your main_reader() function defined outside of the Application class as:

def main_reader(data, val):  # notice it's outside of the Application scope
    sen = minimalmodbus.Instrument(*data["__init__"])  # initialize Instrument
    for k, v in data["serial"].items():  # apply the serial settings
        setattr(sen.serial, k, v)
    sen.mode = data["mode"]  # set the mode
    try:
        read = sen.read_registers(0, 42)
    except OSError:
        read = "Communication Error"
    except ValueError:
        read = "RTU Error"
    print(read)

It should stop throwing errors. Also, you've used threading.Lock in your original code - I don't know what you were trying achieve with it, but it most certainly doesn't do what you think it does.