Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Problem using greenlet to execute multiple functions simultaneously

The following script has the purpose of execute many functions simultaneously, but i don't have idea why it's not working correctly.
The functions are executed in sequential way, not parallel.

I would appreciate any suggestion to clarify me, about what i'm doing wrong.

import time
import eventlet


EXECUTION_TIMEOUT = 10

def example(name, steps_limit):
    print 'Starting process %s with %d steps' % (name, steps_limit)
    for i in range(1, steps_limit+1):
        print "Process %s, step %d" % (name, i)
        time.sleep(2)
    print 'Finishing process %s with %d steps' % (name, steps_limit)


def fetch(input_data):
    example(input_data['name'], input_data['steps'])

test_data = [{'name':'proceso1', 'steps':3},
             {'name':'proceso2', 'steps':5},
             {'name':'proceso3', 'steps':6},
             ]

def main():
    #Setting up time out
    timeout = eventlet.timeout.Timeout(EXECUTION_TIMEOUT)
    #initialize pool
    pool = eventlet.GreenPool(size=1000)
    try:
        for hits in pool.imap(fetch, test_data):
            pass
    except eventlet.Timeout:
        print 'Operation interrupted by timeout concept'
    finally:
        timeout.cancel()


if __name__ == '__main__':
    main()
like image 508
rmaceissoft Avatar asked Nov 04 '22 18:11

rmaceissoft


1 Answers

You're right about time.sleep() being the culprit.

But you don't have to replace time.sleep() with eventlet.sleep(). Instead, you can monkey patch time.sleep to become eventlet.sleep.

This is, in fact, preferred, because that way the libraries that you did not know use time.sleep also get fixed whereas if you fix it yourself you only fixed your code.

For this reason, in gevent, we just recommend to call gevent.monkey.patch_all() for all programs. In eventlet, there's a similar function exists.

like image 72
Denis Avatar answered Nov 09 '22 15:11

Denis