Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to deal with multiple serial ports for R/W using twisted?

Going through the twisted finger tutorial and seen the SO questions:

  • Question-1
  • Question-2

However, I can't (yet) write a twisted program that can read & write from multiple serial ports, especially where the protocol involves reading single or multiple lines, and writing back to the device accordingly.

What I am trying to do is open 2 pairs (i.e. total of 4) serial ports, for 2 modems. Communication with modems is using Hayes AT command set. While most of the command/response exchanges with modem is via the command-port, there are few diagnostic information that are available only via the diagnostic-port, for each modem. The diagnostic information should lead to a state-machine (device-state, connection-state) to be modified.

Here is a rough skeletal program of what I understand as the potential approach (based on single port examples):

class CommandProtocol(LineOnlyReceiver):
    def connectionMade(self):
        log.msg("Connected to command port")

    def lineReceived(self, line):
        print repr(line)
        processCommandLine(line)

class DiagnosticProtocol(LineOnlyReceiver):
    def connectionMade(self):
        log.msg("Connected to diag port")

    def lineReceived(self, line):
        print repr(line)
        processDiagnosticLine(line)

...

# modem1 ports
cmdPort[0] = SerialPort(CommandProtocol, "/dev/ttyUSB0", reactor, 115200)
diagPort[0] = SerialPort(DiagnosticProtocol, "/dev/ttyUSB1", reactor, 115200)
# modem2 ports
cmdPort[1] = SerialPort(CommandProtocol, "/dev/ttyUSB3", reactor, 115200)
diagPort[1] = SerialPort(DiagnosticProtocol, "/dev/ttyUSB4", reactor, 115200)

However, I am at loss, as to how do I do the following:

  • How/where do I accept CLI input from user, that then triggers sending a set of AT command to the modems ?
  • Correlate the information received on command port for ttyUSB0 & ttyUSB1 for modem1, and similarly for the other pair for modem2 ? Note that each modem has it's own state-machine (device-state and connection-state)
  • Does twisted provide any mechanism for management of multiple state-machines by application ?
  • It is possible that the USB-serial connection to the modem is destroyed due to the modem being unplugged, and re-established on being plugged back-in. How can I detect such events and add the monitoring of the corresponding device-ports to the reactor ? Currently, I'm doing it statically in the main application.
like image 903
bdutta74 Avatar asked May 22 '15 13:05

bdutta74


1 Answers

Note on your example code

I don't see you instantiating your classes before registering them to the reactor. I expect that will fail badly. Here is a similar snippet of running code of mine:

# stuff to process messages coming from the serial port
class SerialEater(basic.LineReceiver):
    statusCallback = None

    def __init__(self):
        self.keyinprocess = None

    def lineReceived(self, data):
      self.dealWithSerial(data)

    def connectionLost(self, reason):
      if(reactor.running):
        print "Serial lost but reactor still running! reason: " + str(reason) + " at time " + time.asctime()
    [...etc...]



# Register the serialport into twisted
serialhandler = SerialEater()                   # <------------- instantiate
SerialPort(serialhandler, '/dev/ttyUSB0', reactor, baudrate=115200)

How/where do I accept CLI input from user, that then triggers sending a set of AT command to the modems ?

Much like how you can register Serial handlers into Twisted, you can register handlers for standard io, for instance:

# stuff to pull cbreak char input from stdin
class KeyEater(basic.LineReceiver):

    def __init__(self):
      self.setRawMode() # Switch from line mode to "however much I got" mode

    def connectionLost(self, reason):
      if(reactor.running):
        self.sendLine( "Keyboard lost but reactor still running! reason: " + str(reason) + " at time " + time.asctime())

    def rawDataReceived(self, data):
      key = str(data).lower()[0]
      try:
        if key == '?':
          key = "help"
     [...etc...]

# register the stdio handler into twisted
keyboardobj = KeyEater()
keyboardobj.serialobj = serialhandler
stdio.StandardIO(keyboardobj,sys.stdin.fileno())

Correlate the information received on command port for ttyUSB0 & ttyUSB1 for modem1, and similarly for the other pair for modem2 ? Note that each modem has it's own state-machine (device-state and connection-state)

In normal use, each connection-instance is going to have its own state machine (wrapped up in the instance of the class that you register into the reactor along with the connection).

You as the programmer choose how you want to connect the states of the classes, but often its via pushing reference to the partner classes.

Below, this answer contains runnable code that will illustrate how data is connected between state-machines/interface. This is also illustrated in this SO: Persistent connection in twisted


Does twisted provide any mechanism for management of multiple state-machines by application ?

If by "application" you mean "your twisted code" then then the answer is absolutely YES!

The typical Twisted app is an array of state-machines, all with some amazingly well defined interfaces. I started my Twisted adventure intending to write an app with two state-machines (a serial and keyboard), but when I became comfortable with twisted was doing I realized it was trivial to add on extra interfaces and state-machines (through all the wonder of the tx libraries). All in one afternoon I added on a rough web interface, a websocket interface, then laid SSL over both and even added on an SSH debug interface. Once you get a rolling, adding interfaces and state-machines become trivial.

In many (all?) cases, the twisted model is that a state-machine will reside in an instantiated class that is tied to a connection and that has been registered into the (one-and-only-one) main event-loop.

With connection types that spawn off new state-machines (think http connections) you register one factory-class/state-machine along with the listening connection which together enable the app of spawning off new classes/state-machines for each new connection. Twisted applications routinely 10s or even 100s of thousands of concurrent instances of state when run at scale.

Twisted is amazing if your trying to glue together different protocols and states (... with all of it being in a event loop of your choice (select/epoll/kqueue/etc))

The following is runnable sample code that should illustrate many of these points. Read the comments before def main() for more background on the code:

#!/usr/bin/python
#
# Frankenstein-esk amalgam of example code
#   Key of which comes from the Twisted "Chat" example
#   (such as: http://twistedmatrix.com/documents/12.0.0/core/examples/chatserver.py)

import sys # so I can get at stdin
import os # for isatty
import termios, tty # access to posix IO settings
from random import random
from twisted.internet import reactor
from twisted.internet import stdio # the stdio equiv of listenXXX
from twisted.protocols import basic # for lineReceiver for keyboard
from twisted.internet.protocol import Protocol, ServerFactory

class MyClientConnections(basic.LineReceiver):

    def __init__(self):
        self.storedState = "Idle"
        self.connectionpos = None

    def connectionMade(self):
        self.factory.clients.append(self) # <--- magic here :
            # protocol automagically has a link to its factory class, and
            # in this case that is being used to push each new connection
            # (which is in the form of this class) into a list that the
            # factory can then access to get at each of the connections
        self.connectionpos = str(self.factory.clients.index(self)) # figure out 
                                      # where I am in the connection array
        print "Got new client! (index:", self.connectionpos + ")"
        self.transport.write("---\nYour connection: " + self.connectionpos + "\n---\n")

    def connectionLost(self, reason):
        print "Lost a client!"
        self.factory.clients.remove(self)

    # used to pretend that something was typed on a telnet connection
    def fakeInput(self, message):
        self.transport.write("FAKING Input: '" + message + "'\n")
        self.lineReceived(message)

    #this is only in a def on its own so I can lump my demo callLater
    def stateUpdate(self, newState, delay):
        self.storedState = newState
        # the following is a hack to fake data coming in this interface
        reactor.callLater(delay, self.fakeInput, newState + " DONE")

    def processInput(self, newState):
        # all the logic in here is junk to make a demo, real code may or may-not look like
        # this.  This junk logic is an example statemachine though
        if self.storedState == "Idle":
            if newState == "start":
                self.stateUpdate("State A", 1)        
                # send a message to this connection
                self.transport.write("starting state machine\n")
                # send a message to the term in which the script it running
                print "Connection [" + self.connectionpos + "] starting state machine"
        elif self.storedState == "State A":
            if newState == "State A DONE":
                self.transport.write("Beginning state B\n")
                self.stateUpdate("State B", 2)
        elif self.storedState == "State B":
            if newState == "State B DONE":
                self.transport.write("Beginning state C\n")
                self.stateUpdate("State C", 2)
        elif self.storedState == "State C":
            if newState == "State C DONE":
                self.storedState = "Idle"
                # send a message to this connection
                self.transport.write("Returning to Idle state\n")
                # send a message to the term in which the script it running
                print "Connection [" + self.connectionpos + "] return to Idle state"

    def lineReceived(self, line):
        # print "received '" + line +"' from connection", self.factory.clients.index(self)
        self.processInput(line)

class MyServerFactory(ServerFactory):
    protocol = MyClientConnections

    def __init__(self):
        self.clients = [] # this gets filled from the class above

    def sendToAll(self, message):
      for c in self.clients:  # Read MyClientConnections class for background
        c.transport.write(message)

    def randStart(self, width):
      for c in self.clients:
        startDelay = random() * width
        print "Starting client " + str(c.connectionpos) + " in " +str(startDelay) + " secs" 
        reactor.callLater(startDelay, c.processInput, "start")

# to set keyboard into cbreak mode -- just because I like it that way...
class Cbreaktty(object):
    org_termio = None
    my_termio = None

    def __init__(self, ttyfd):
        if(os.isatty(ttyfd)):
            self.org_termio = (ttyfd, termios.tcgetattr(ttyfd))
            tty.setcbreak(ttyfd)
            print '  Set cbreak mode'
            self.my_termio = (ttyfd, termios.tcgetattr(ttyfd))
        else:
          raise IOError #Not something I can set cbreak on!

    def retToOrgState(self):
        (tty, org) = self.org_termio
        print '  Restoring terminal settings'
        termios.tcsetattr(tty, termios.TCSANOW, org)


class KeyEater(basic.LineReceiver):

    def __init__(self, factoryObj):
        self.setRawMode() # Switch from line mode to "however much I got" mode
        # the following is one of the key connecting ideas in twisted, the object
        # that contains another state machine (really all of the tcp statemachines)
        # has been passed into this class via its init.
        self.factoryObj = factoryObj

    def rawDataReceived(self, data):
        key = str(data).lower()[0]
        if key == 's':
            # The following line is going to call (from within the factory object)
            # the random start def
            self.factoryObj.randStart(5)
        elif key == 'd':
            print "State Dump of connections"
            print "-------------------------"
            for c in self.factoryObj.clients:
                print "#" + str(c.connectionpos) + "      " + c.storedState
        elif key == 'q':
            reactor.stop()
        else:
            print "--------------"
            print "  If you haven't already, connect to this script via a"
            print "  'telnet localhost 5000' at least one (multiple connections"
            print "  are better)"
            print "Press:"
            print "      s  - randomly start all clients"
            print "      d  - dump the state of all connected clients"
            print "      q  - to cleanly shutdown"
            print " Note: you can type commands in the connections, things"
            print "       most useful of which is 'start'"
            print "---------------"

# Custom tailored example for SO:30397425
# 
# This code is a mishmash of styles and techniques. Both to provide different examples of how
# something can be done and because I'm lazy.  Its been built and tested on OSX and linux,
# it should be portable (other then perhaps termal cbreak mode).  If you want to ask
# questions about this code contact me directly via mail to mike at partialmesh.com
#
# While it isn't directly using serial ports, the tcp connections that its using are a good
# parallel.
#
# It should be used by running the script and then opening up many windows telnet'ing into
# localhost 5000.
#
# Once running press any key in the window where the script was run and it will give
# instructions.  
# The normal use case would be to type "s" to queue statemachine
# start-ups, then repeatedly press 'd' to dump the status of all the state machines
#
# 'start' can be typed into any of the telnet connections to start them by hand too.


def main():
    client_connection_factory = MyServerFactory()

    try:
      termstate = Cbreaktty(sys.stdin.fileno())
    except IOError:
      sys.stderr.write("Error: " + sys.argv[0] + " only for use on interactive ttys\n")
      sys.exit(1)

    keyboardobj = KeyEater(client_connection_factory)

    stdio.StandardIO(keyboardobj,sys.stdin.fileno())
    reactor.listenTCP(5000, client_connection_factory)
    reactor.run()
    termstate.retToOrgState()
if __name__ == '__main__':
  main()

It is possible that the USB-serial connection to the modem is destroyed due to the modem being unplugged, and re-established on being plugged back-in. How can I detect such events and add the monitoring of the corresponding device-ports to the reactor ? Currently, I'm doing it statically in the main application.

After research I don't have an easy answer. I still suspect that the following logic will be close to a solution but I didn't have any luck finding code that implements this today.

My guess is there will be a reasonable way to figure out if a USB event has occurred, and work out if a serial device has been added. But I doubt there will a good way to figure out if it is one of your serial devices - much less if its your Command or Diagnostic interface (Unless your building hardware and can control the USB IDs of the devices)

Events are fired on errors with serial ports (at least from my experience on linux), but I'm unsure how/where a USB unplug would register.


Other links that might be of use to you

  • Twisted implementation of Utilities for talking to a GSM modem over USB via AT commands: https://github.com/smn/txgsm
  • Twisted implementation of weather station through USB: https://gist.github.com/claws/2464017
like image 129
Mike Lutz Avatar answered Oct 05 '22 12:10

Mike Lutz