Script blocks on thread when executing a python script, but not in interactive mode

Question

TL;DR: Why would a threaded process run as intended (detached python thread) from interactive mode like myprocess.start() but block on a subthread when run from the shell, like python myprocess.py?

---

Background: I subclassed threading.Thread for my class, which also calls two other Thread-type subclasses. It looks like:

class Node(threading.Thread):
    def __init__(self, gps_device):
        threading.Thread.__init__(self)
        self.daemon = False

        logging.info("Setting up GPS service")
        self.gps_svc = gps.CoordinateService(gps_device)
        self.gps_svc.daemon = True

        logging.info("Setting up BLE scanning service")
        # TODO: This is blocking when run in terminal (aka how we do on Raspberry Pi)
        self.scan_svc = scan.BleMonitor()
        self.scan_svc.daemon = True

        logging.info("Node initialized - ready for start")

    def run(self):
        self.gps_svc.start()
        self.scan_svc.start()  # blocks here in terminal

        do stuff...

The two services (gps_svc and scan_svc) both work as intended from the interpreter in interactive mode like node = Node(...); node.start(). When I invoke the interpreter with a script, the gps_svc starts and functions, but the scan_svc blocks at a specific line where it listens to a Bluetooth device.

BLE Scanner is below (it's long-ish). This is the parent class for BleMonitor - none of the guts are different, I just added a couple of utility functions.

---

Question: Why is this happening? Can I run/interact with a process versus a thread (ie: call methods of the class and get data in real-time)?

class Monitor(threading.Thread):
    """Continously scan for BLE advertisements."""

    def __init__(self, callback, bt_device_id, device_filter, packet_filter):
        """Construct interface object."""
        # do import here so that the package can be used in parsing-only mode (no bluez required)
        self.bluez = import_module('bluetooth._bluetooth')

        threading.Thread.__init__(self)
        self.daemon = False
        self.keep_going = True
        self.callback = callback

        # number of the bt device (hciX)
        self.bt_device_id = bt_device_id
        # list of beacons to monitor
        self.device_filter = device_filter
        self.mode = get_mode(device_filter)
        # list of packet types to monitor
        self.packet_filter = packet_filter
        # bluetooth socket
        self.socket = None
        # keep track of Eddystone Beacon <-> bt addr mapping
        self.eddystone_mappings = []

    def run(self):
        """Continously scan for BLE advertisements."""
        self.socket = self.bluez.hci_open_dev(self.bt_device_id)

        filtr = self.bluez.hci_filter_new()
        self.bluez.hci_filter_all_events(filtr)
        self.bluez.hci_filter_set_ptype(filtr, self.bluez.HCI_EVENT_PKT)
        self.socket.setsockopt(self.bluez.SOL_HCI, self.bluez.HCI_FILTER, filtr)

        self.toggle_scan(True)

        while self.keep_going:
            pkt = self.socket.recv(255)
            event = to_int(pkt[1])
            subevent = to_int(pkt[3])
            if event == LE_META_EVENT and subevent == EVT_LE_ADVERTISING_REPORT:
                # we have an BLE advertisement
                self.process_packet(pkt)

    def toggle_scan(self, enable):
        """Enable and disable BLE scanning."""
        if enable:
            command = "\x01\x00"
        else:
            command = "\x00\x00"
        self.bluez.hci_send_cmd(self.socket, OGF_LE_CTL, OCF_LE_SET_SCAN_ENABLE, command)

    def process_packet(self, pkt):
        """Parse the packet and call callback if one of the filters matches."""
        # check if this could be a valid packet before parsing
        # this reduces the CPU load significantly
        if (self.mode == MODE_BOTH and \
                (pkt[19:21] != b"\xaa\xfe") and (pkt[19:23] != b"\x4c\x00\x02\x15")) \
                or (self.mode == MODE_EDDYSTONE and (pkt[19:21] != b"\xaa\xfe")) \
                or (self.mode == MODE_IBEACON and (pkt[19:23] != b"\x4c\x00\x02\x15")):
            return

        bt_addr = bt_addr_to_string(pkt[7:13])
        rssi = bin_to_int(pkt[-1])
        # strip bluetooth address and parse packet
        packet = parse_packet(pkt[14:-1])

        # return if packet was not an beacon advertisement
        if not packet:
            return


        # we need to remember which eddystone beacon has which bt address
        # because the TLM and URL frames do not contain the namespace and instance
        self.save_bt_addr(packet, bt_addr)
        # properties hold the identifying information for a beacon
        # e.g. instance and namespace for eddystone; uuid, major, minor for iBeacon
        properties = self.get_properties(packet, bt_addr)

        if self.device_filter is None and self.packet_filter is None:
            # no filters selected
            self.callback(bt_addr, rssi, packet, properties)

        elif self.device_filter is None:
            # filter by packet type
            if is_one_of(packet, self.packet_filter):
                self.callback(bt_addr, rssi, packet, properties)
        else:
            # filter by device and packet type
            if self.packet_filter and not is_one_of(packet, self.packet_filter):
                # return if packet filter does not match
                return

            # iterate over filters and call .matches() on each
            for filtr in self.device_filter:
                if isinstance(filtr, BtAddrFilter):
                    if filtr.matches({'bt_addr':bt_addr}):
                        self.callback(bt_addr, rssi, packet, properties)
                        return

                elif filtr.matches(properties):
                    self.callback(bt_addr, rssi, packet, properties)
                    return

    def save_bt_addr(self, packet, bt_addr):
        """Add to the list of mappings."""
        if isinstance(packet, EddystoneUIDFrame):
            # remove out old mapping
            new_mappings = [m for m in self.eddystone_mappings if m[0] != bt_addr]
            new_mappings.append((bt_addr, packet.properties))
            self.eddystone_mappings = new_mappings

    def get_properties(self, packet, bt_addr):
        """Get properties of beacon depending on type."""
        if is_one_of(packet, [EddystoneTLMFrame, EddystoneURLFrame, \
                              EddystoneEncryptedTLMFrame, EddystoneEIDFrame]):
            # here we retrieve the namespace and instance which corresponds to the
            # eddystone beacon with this bt address
            return self.properties_from_mapping(bt_addr)
        else:
            return packet.properties

    def properties_from_mapping(self, bt_addr):
        """Retrieve properties (namespace, instance) for the specified bt address."""
        for addr, properties in self.eddystone_mappings:
            if addr == bt_addr:
                return properties
        return None

    def terminate(self):
        """Signal runner to stop and join thread."""
        self.toggle_scan(False)
        self.keep_going = False
        self.join()

Answer 1

From the Python Documentation, I am taking it the interpreter in interactive mode is in violation of the following when it comes to threads:

In CPython, the global interpreter lock, or GIL, is a mutex that protects access to Python objects, preventing multiple threads from executing Python bytecodes at once. This lock is necessary mainly because CPython's memory management is not thread-safe. (However, since the GIL exists, other features have grown to depend on the guarantees that it enforces.)

Therefore, the rule exists that only the thread that has acquired the GIL may operate on Python objects or call Python/C API functions. In order to emulate concurrency of execution, the interpreter regularly tries to switch threads (see sys.setswitchinterval()). The lock is also released around potentially blocking I/O operations like reading or writing a file, so that other Python threads can run in the meantime.

I would need to look into this further, but my suspicions point me to a conflict between the GIL and the threaded object management. Hope that helps or someone has more to add.