Why is `gevent.spawn` different than a monkeypatched `threading.Thread()`?

Question

While double checking that threading.Condition is correctly monkey patched, I noticed that a monkeypatched threading.Thread(…).start() behaves differently from gevent.spawn(…).

Consider:

from gevent import monkey; monkey.patch_all()
from threading import Thread, Condition
import gevent

cv = Condition()

def wait_on_cv(x):
    cv.acquire()
    cv.wait()
    print "Here:", x
    cv.release()

# XXX: This code yields "This operation would block forever" when joining the first thread
threads = [ gevent.spawn(wait_on_cv, x) for x in range(10) ]

"""
# XXX: This code, which seems semantically similar, works correctly
threads = [ Thread(target=wait_on_cv, args=(x, )) for x in range(10) ]
for t in threads:
    t.start()
"""

cv.acquire()
cv.notify_all()
print "Notified!"
cv.release()

for x, thread in enumerate(threads):
    print "Joining", x
    thread.join()

Note, specifically, the two comments starting with XXX.

When using the first line (with gevent.spawn), the first thread.join() raises an exception:

Notified!
Joining 0
Traceback (most recent call last):
  File "foo.py", line 30, in 
    thread.join()
  File "…/gevent/greenlet.py", line 291, in join
    result = self.parent.switch()
  File "…/gevent/hub.py", line 381, in switch
    return greenlet.switch(self)
gevent.hub.LoopExit: This operation would block forever

However, Thread(…).start() (the second block), everything works as expected.

Why would this be? What's the difference between gevent.spawn() and Thread(…).start()?

Answer 1

What happen in your code is that the greenlets that you have created in you threads list didn't have yet the chance to be executed because gevent will not trigger a context switch until you do so explicitly in your code using gevent.sleep() and such or implicitly by calling a function that block e.g. semaphore.wait() or by yielding and so on ..., to see that you can insert a print before cv.wait() and see that it's called only after cv.notify_all() is called:

def wait_on_cv(x):
    cv.acquire()
    print 'acquired ', x
    cv.wait()
    ....

So an easy fix to your code will be to insert something that will trigger a context switch after you create your list of greenlets, example:

...
threads = [ gevent.spawn(wait_on_cv, x) for x in range(10) ]
gevent.sleep()  # Trigger a context switch
...

Note: I am still new to gevent so i don't know if this is the right way to do it :)

This way all the greenlets will have the chance to be executed and each one of them will trigger a context switch when they call cv.wait() and in the mean time they will register them self to the condition waiters so that when cv.notify_all() is called it will notify all the greenlets.

HTH,