Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to multiplex multiple blocking Python generators into one?

Consider the following pseudo code:

def g_user():
    while True:
        yield read_user_input()

def g_socket():
    while True:
        yield read_socket_input()

def g_combined(gu, gs):
    # should read user or socket input, whichever is available

    while True:
        sel = select(gu, gs)
        if sel.contains(gu):
            yield gu.next()
        if sel.contains(gs):
            yield gs.next()

gc = g_combined ( g_user(), g_socket() )

How to implement this in easiest way?

like image 328
Vi. Avatar asked Feb 08 '12 01:02

Vi.


1 Answers

Look's like someone already implemented this: http://www.dabeaz.com/generators/genmulti.py

Mirrored here:

import Queue, threading
def gen_multiplex(genlist):
    item_q = Queue.Queue()
    def run_one(source):
        for item in source: item_q.put(item)

    def run_all():
        thrlist = []
        for source in genlist:
            t = threading.Thread(target=run_one,args=(source,))
            t.start()
            thrlist.append(t)
        for t in thrlist: t.join()
        item_q.put(StopIteration)

    threading.Thread(target=run_all).start()
    while True:
        item = item_q.get()
        if item is StopIteration: return
        yield item
like image 173
Vi. Avatar answered Sep 21 '22 14:09

Vi.