How does cherrypy handle user threads?

Question

I'm working on a django app right and I'm using cherrypy as the server. Cherrypy creates a new thread for every page view. I'd like to be able to access all of these threads (threads responsible for talking to django) from within any of them. More specifically I'd like to be able to access the thread_data for each of these threads from within any of them. Is this possible? If so, how do I do it?

Answer 1

CherryPy's wsgiserver doesn't create a new thread for every request--it uses a pool. Each of those worker threads is a subclass of threading.Thread, so all of them should be accessible via threading.enumerate().

However, if you're talking specifically about cherrypy.thread_data, that's something else: a threading.local. If you're using a recent version of Python, then all that's coded in C and you (probably rightfully) don't have cross-thread access to it from Python. If you really need it and really know what you're doing, the best technique is usually to stick an additional reference to such things in a global container at the same time that they are inserted into the thread_data structure. I recommend dicts with weakrefs as keys for those global containers--there are enough Python ORM's that use them for connection pools (see my own Geniusql, for example) that you should be able to learn how to implement them fairly easily.

Answer 2

My first response to a question like this isn't to tell you how to do it but to stress that you really should reconsider before moving forward with this. I normally shy away from threaded web-servers, in favor of multi-process or asynchronous solutions. Adding explicit inter-thread communication to the mix only increases those fears.

When a question like this is asked, there is a deeper goal. I suspect that what you think inter-thread communication would solve can actually be solved in some other, safer way.