twisted: one client, many servers

Question

I'm trying to use twisted to create a cluster of computers that run one program on a piece of a larger dataset.

My "servers" receive a chunk of data from the client and run command x on it.

My "client" connects to multiple servers giving them each a chunk of data and telling them what parameters to run command x with.

My question is: is there a way to set up the reactor loop to connect to many servers:

reactor.connectTCP('localhost', PORT, BlastFactory())
reactor.run()

or do I have to swap client and server in my paradigm?

Answer 1

Just call connectTCP multiple times.

The trick, of course, is that reactor.run() blocks "forever" (the entire run-time of your program) so you don't want to call that multiple times.

You have several options; you can set up a timed call to make future connections, or you can start new connections from events on your connection (like connectionLost or clientConnectionFailed).

Or, at the simplest, you can just set up multiple connection attempts before reactor.run() kicks off the whole show, like this:

for host in hosts:
    reactor.connectTCP(host, PORT, BlastFactory())
reactor.run()