Python script hanging when running in the background

Question

I have a Python script (run on 2.7) that behaves differently when I run it in from the command line versus the background. When I run it from the terminal it runs as expected, the two threads run as daemons writing output to the window while the main loop waits for the quit command. It runs forever until I enter quit:

python test.py

When the same program is run in the background, both threads run one time and then the program hangs (I've narrowed it down to the raw_input, I guess I made a incorrect assumption that the two threads would continue on even if run in the background and raw_input blocked the main one. e.g the two threads would basically run forever since there is no input in this scenario).

python test.py &

My goal is to have one program with those loops running (potentially forever) but if I ran it from the terminal would accept input.

In order to allow the program to run both from the terminal / in the background do I need to basically put an if statement before the raw_input that checks whether it's in the background or not or am I missing else that would help?

import sys
import time
from threading import Thread

def threadOne():
    while True:
        print("Thread 1")
        time.sleep(1)

def threadTwo():
    while True:
        print("Thread 2")
        time.sleep(1)

# Run the threads in the background as daemons
threadOne = Thread(target = threadOne)
threadOne.daemon = True
threadOne.start()

threadTwo = Thread(target = threadTwo)
threadTwo.daemon = True
threadTwo.start()

# Main input loop.  This will allow us to enter input.  The
# threads will run forever unless "quit" is entered.  This
# doesn't run when the program is run in the background (I
# incorrectly assumed it would just run forever with no input 
# ever being entered in that scenario).
while True:
    userInput = ""
    userInput = raw_input("")
    time.sleep(1)

    # This should allow us to exit out
    if str(userInput) == "quit":
        sys.exit()

Answer 1

In order to allow the program to run both from the terminal / in the background do I need to basically put an if statement before the raw_input that checks whether it's in the background or not or am I missing else that would help?

In a way this probably works (I am assuming you are running this on a *nix), however if user were to send the process backing into background (i.e. suspend it using CtrlZ then resuming it in background with %&) while raw_input is waiting on user input, then the read on stdin will then be blocked as it is in the background, thus causing the kernel to stop the process as this is how stdio works. If this is acceptable (basically user has to hit enter before suspending the process), you can simply do this:

import os

while True:
    userInput = ""
    if os.getpgrp() == os.tcgetpgrp(sys.stdout.fileno()):
        userInput = raw_input("")
    time.sleep(1)

What os.getpgrp does is that returns the id of the current os group, and then os.tcgetpgrp gets the process group associated with the stdout for this process, if they match, it means this process is currently in the foreground, meaning you can probably call raw_input without blocking the threads.

Another question raised a similar issue and I have a longer explanation at: Freeze stdin when in the background, unfreeze it when in the foreground.

---

The better way is to combine this with select.poll, and address interactive I/O separately (by using /dev/tty directly) from standard I/O as you don't want stdin/stdout redirection being "polluted" by that. Here is the more complete version that contains both these ideas:

tty_in = open('/dev/tty', 'r')
tty_out = open('/dev/tty', 'w')
fn = tty_in.fileno()
poll = select.poll()
poll.register(fn, select.POLLIN)

while True:
    if os.getpgrp() == os.tcgetpgrp(fn) and poll.poll(10):  # 10 ms
        # poll should only return if the input buffer is filled,
        # which is triggered when a user enters a complete line,
        # which lets the following readline call to not block on
        # a lack of input.
        userInput = tty_in.readline()
        # This should allow us to exit out
        if userInput.strip() == "quit":
            sys.exit()

The background/foreground detection is still needed as the process is not fully detached from the shell (since it can be brought back to the foreground) thus poll will return the fileno of the tty if any input is sent into the shell, and if this triggers the readline which will then stop the process.

This solution has the advantage of not requiring the user to hit enter and quickly suspend the task to send it back to the background before raw_input traps and blocks stdin to stop the process (as poll checks whether there's input to be read), and allow proper stdin/stdout redirection (as all interactive input is handled via /dev/tty) so users can do something like:

$ python script.py < script.py 2> stderr
input stream length: 2116

In the completed example below it also provide a prompt to the user, i.e. a > is shown whenever a command is sent or whenever the process is returned to foreground, and wrapped the entire thing in a main function, and modified the second thread to spit things out at stderr:

import os
import select
import sys
import time
from threading import Thread

def threadOne():
    while True:
        print("Thread 1")
        time.sleep(1)

def threadTwo():
    while True:
        # python 2 print does not support file argument like python 3,
        # so writing to sys.stderr directly to simulate error message.
        sys.stderr.write("Thread 2\n")
        time.sleep(1)

# Run the threads in the background
threadOne = Thread(target = threadOne)
threadOne.daemon = True

threadTwo = Thread(target = threadTwo)
threadTwo.daemon = True

def main():
    threadOne.start()
    threadTwo.start()

    tty_in = open('/dev/tty', 'r')
    tty_out = open('/dev/tty', 'w')
    fn = tty_in.fileno()
    poll = select.poll()
    poll.register(fn, select.POLLIN)

    userInput = ""
    chars = []
    prompt = True

    while True:
        if os.getpgrp() == os.tcgetpgrp(fn) and poll.poll(10):  # 10 ms
            # poll should only return if the input buffer is filled,
            # which is triggered when a user enters a complete line,
            # which lets the following readline call to not block on
            # a lack of input.
            userInput = tty_in.readline()
            # This should allow us to exit out
            if userInput.strip() == "quit":
                sys.exit()
            # alternatively an empty string from Ctrl-D could be the
            # other exit method.
            else:
                tty_out.write("user input: %s\n" % userInput)
                prompt = True
        elif not os.getpgrp() == os.tcgetpgrp(fn):
            time.sleep(0.1)
            if os.getpgrp() == os.tcgetpgrp(fn):
                # back to foreground, print a prompt:
                prompt = True

        if prompt:
            tty_out.write('> ')
            tty_out.flush()
            prompt = False

if __name__ == '__main__':
    try:
        # Uncomment if you are expecting stdin
        # print('input stream length: %d ' % len(sys.stdin.read()))
        main()
    except KeyboardInterrupt:
        print("Forcibly interrupted.  Quitting")
        sys.exit()  # maybe with an error code

Has been an interesting exercise; this was a rather good and interesting question, if I may say.

One final note: this is not cross-platform, it will not work on Windows as it doesn't have select.poll and /dev/tty.