Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Run multiple functions every second, write result to file

I'm trying to run three functions (each can take up to 1 second to execute) every second. I'd then like to store the output from each function, and write them to separate files.

At the moment I'm using Timers for my delay handling. (I could subclass Thread, but that's getting a bit complicated for this simple script)

def main:
    for i in range(3):
        set_up_function(i)
        t = Timer(1, run_function, [i])
        t.start()
    time.sleep(100) # Without this, main thread exits

def run_function(i):
    t = Timer(1, run_function, [i])
    t.start()
    print function_with_delay(i)

What's the best way to handle the output from function_with_delay? Append the result to a global list for each function?

Then I could put something like this at the end of my main function:

...
while True:
    time.sleep(30) # or in a try/except with a loop of 1 second sleeps so I can interrupt
    for i in range(3):
        save_to_disk(data[i])

Thoughts?


Edit: Added my own answer as a possibility

like image 732
Alex L Avatar asked Jan 16 '12 03:01

Alex L


1 Answers

I believe the python Queue module is designed for precisely this sort of scenario. You could do something like this, for example:

def main():
    q = Queue.Queue()
    for i in range(3):
        t = threading.Timer(1, run_function, [q, i])
        t.start()

    while True:
        item = q.get()
        save_to_disk(item)
        q.task_done()

def run_function(q, i):
    t = threading.Timer(1, run_function, [q, i])
    t.start()
    q.put(function_with_delay(i))
like image 168
srgerg Avatar answered Nov 14 '22 23:11

srgerg