Make a twister server take initiative

Question

I have a server in twisted, implementing a LineReceiver protocol. When I call sendLine in response to a client message, it writes the line to the client immediately, as one would expect.

But say the client asks the server to do a lengthy calculation. I want the server to periodically send a progress message to the client. When the server takes initiative and calls sendLine without the client having asked for anything, it seems to wait for the client to send a message to the server before sending anything.

How do I send a message from the server to the client immediately, without having the client explicitly ask for it?

Answer 1

Use deferreds if you perform calculation asynchronously.
Other way if it's some long calculation in separate Thread, started by lets say deferrToThread(), use reactor.callFromThread()
(I assume we don't do heavy calculation in main loop - that's very, very wrong :)) little example:

def some_long_foo(data_array, protocol):
    def send_msg(msg, protocol):
        # It actually looks petter in classes without pushing protocol here and
        # there
        protocol.transport.write(msg)

    for n, chunk in enumerate(data_array):
        do_something_cool(chunk)
        if n and (n % 10 == 0):
            from twisted.internet import reactor
            # here send_msg will be safely executed in main reactor loop
            reactor.callFromThread(send_msg, '10 more chunks processed',
                                   protocol)

# Somwhere in lineReceived we start long calculation
def cb(result):
    self.transport.write('got result: {}'.format(result))
d = threads.deferToThread(some_long_foo, data_array, self)
d.addCallback(cb)

Thus now we'll notify client about processing every 10 chunks of data, and then finally send him result. code may be little incorrect, it's just e.g.

docs

UPD: just for clarification: missed sendLine part. Generally it doesn't matter, call it insted of transport.write()