How do Twisted python Factory and Protocol interfaces work?

Question

I am learning Twisted, and beginner's tutorial often uses Factory and Protocol as examples. It appears that Factory and Protocol interfaces do not support sending messages. Is it expected that sending messages are implemented independently of Protocol interface?

class IProcessProtocol(Interface):
    def makeConnection(process):
    def childDataReceived(childFD, data):
    def childConnectionLost(childFD):
    def processExited(reason):
    def processEnded(reason):

Answer 1

See:

• http://twistedmatrix.com/documents/current/api/twisted.internet.protocol.Factory.html
• http://twistedmatrix.com/documents/current/api/twisted.internet.protocol.html

Factories create Protocol instances.

What this means is that the factory will use the protocol to figure out how it should listen and send data (see here and also note: you can also write your own protocol).

These are the methods available to Protocol:

Method  logPrefix Return a prefix matching the class name, to identify log messages related to this protocol instance.
Method  dataReceived  Called whenever data is received.
Method  connectionLost  Called when the connection is shut down.

Inherited from BaseProtocol:

Method  makeConnection  Make a connection to a transport and a server.
Method  connectionMade  Called when a

connection is made.

And once the connection has been made we could do something like write data to transport:

from twisted.internet.protocol import Protocol
class SomeProtocol(Protocol):
    def dataReceived(self, data):
        print('Do something with data: {}'.format(data))

    def connectionMade(self):
        self.transport.write("Hello there")

But wait where does the Protocol get self.transport from?

>>> from twisted.internet.protocol import Protocol, BaseProtocol
>>> import inspect
>>> from pprint import pprint
>>> pprint(inspect.getclasstree(inspect.getmro(Protocol)))
[(<class 'object'>, ()),
 [(<class 'twisted.internet.protocol.BaseProtocol'>, (<class 'object'>,)),
  [(<class 'twisted.internet.protocol.Protocol'>,
    (<class 'twisted.internet.protocol.BaseProtocol'>,))]]]
>>> dir(Protocol)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', 
 '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', 
 '__hash__', '__implemented__', '__init__', '__le__', '__lt__', 
 '__module__', '__ne__', '__new__', '__providedBy__', '__provides__', 
 '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', 
 '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 
 'connected', 'connectionLost', 'connectionMade', 'dataReceived', 
 'logPrefix', 'makeConnection', 'transport']

Okay so Protocol has a transport object/method what about BaseProtocol:

>>> dir(BaseProtocol)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__',           
 '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', 
 '__hash__', '__implemented__', '__init__', '__le__', '__lt__', 
 '__module__', '__ne__', '__new__', '__providedBy__', '__provides__', 
 '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', 
 '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 
 'connected', 'connectionMade', 'makeConnection', 'transport']
>>> type(BaseProtocol.transport)
<class 'NoneType'>

It does by why is it None?

So lets look at BaseProtocol here:

def makeConnection(self, transport): (source) overridden in

twisted.internet.endpoints._WrapIProtocol,
twisted.protocols.amp.BinaryBoxProtocol,
twisted.protocols.basic.NetstringReceiver,
twisted.protocols.ftp.ProtocolWrapper,
twisted.protocols.ftp.SenderProtocol,
twisted.protocols.policies.ProtocolWrapper,
twisted.protocols.stateful.StatefulProtocol Make a connection to a
transport and a server.

Note:

This sets the 'transport' attribute of this
Protocol, and calls the connectionMade() callback.

So when makeConnection is called it sets the transport attribute of the protocol.

So how does that work with the factory?

Let's look at a Factory here and the source for buildProtocol

def buildProtocol(self, addr):
    """
    Create an instance of a subclass of Protocol.

    The returned instance will handle input on an incoming server
    connection, and an attribute "factory" pointing to the creating
    factory.

    Alternatively, C{None} may be returned to immediately close the
    new connection.

    Override this method to alter how Protocol instances get created.

    @param addr: an object implementing L{twisted.internet.interfaces.IAddress}
    """
    p = self.protocol()
    p.factory = self
    return p

Okay, so:

class BaseProtocol:
    """
    This is the abstract superclass of all protocols.

    Some methods have helpful default implementations here so that they can
    easily be shared, but otherwise the direct subclasses of this class are more
    interesting, L{Protocol} and L{ProcessProtocol}.
    """
    connected = 0
    transport = None

    def makeConnection(self, transport):
        """Make a connection to a transport and a server.

        This sets the 'transport' attribute of this Protocol, and calls the
        connectionMade() callback.
        """
        self.connected = 1
        self.transport = transport
        self.connectionMade()

So transport is defined as None here, but still where is transport coming from?

Its coming from reactor when the reactor.connect method is called.

Lets look at a TCP example:

from twisted.internet import reactor
# 
#
#
reactor.connectTCP('localhost', 80, SomeProtocolFactory())

From reactor we call connectTCP like this:

from twisted.internet.iocpreactor import tcp, udp
#
#
#
def connectTCP(self, host, port, factory, timeout=30, bindAddress=None):
    """
    @see: twisted.internet.interfaces.IReactorTCP.connectTCP
    """
    c = tcp.Connector(host, port, factory, timeout, bindAddress, self)
    c.connect()
    return c

Which is calling tcp.Connector like from twisted.internet.iocpreactor import tcp, udp here:

def connect(self):
    """Start connection to remote server."""
    if self.state != "disconnected":
        raise RuntimeError("can't connect in this state")

    self.state = "connecting"
    if not self.factoryStarted:
        self.factory.doStart()
        self.factoryStarted = 1
    ##################
    # ah here we are
    ##################
    self.transport = transport = self._makeTransport()
    if self.timeout is not None:
        self.timeoutID = self.reactor.callLater(self.timeout, transport.failIfNotConnected, error.TimeoutError())
    self.factory.startedConnecting(self)

Which is returning the transport like this:

class Connector(TCPConnector):
    def _makeTransport(self):
        return Client(self.host, self.port, self.bindAddress, self,
                      self.reactor)

Which in turn is creating the socket connection:

• https://github.com/twisted/twisted/blob/trunk/src/twisted/internet/iocpreactor/tcp.py#L244

So the short answer to your question:

Is it expected that sending messages are implemented independently of Protocol interface?

The Protocol initializes transport to None, when the reactor calls connect it sets transport on the Protocol instance.

The reactor then uses the protocols transport object to read/write when incoming/outgoing connections are made.

We are able to send data over a tcp socket with the Protocol instance by using self.transport.write().

See:

• how to send data with twisted protocol via factory