What is the correct way to close a Twisted conch SSH connection?

Question

What is the correct way to close a Twisted conch SSH connection? Is there an explicit way to do this?

All of the Twisted conch examples I have seen close the SSH channel then stop the reactor. The reactor shutdown seems to handle closing the connection. However, I'm using the wxreactor with wxPython and I do not want to stop the reactor, but I want to close the ssh connection when I'm finished with it.

After looking at t.c.s.connection it seemed like the serviceStopped() method was the way to go. It closes all open channels and runs _cleanupGlobalDeferreds() when finished, but then I started getting exceptions like the one below:

Unhandled Error
Traceback (most recent call last):
  File "C:\Users\me\venv\lib\site-packages\twisted\internet\tcp.py", line 203, in doRead
    return self._dataReceived(data)
  File "C:\Users\me\venv\lib\site-packages\twisted\internet\tcp.py", line 209, in _dataReceived
    rval = self.protocol.dataReceived(data)
  File "C:\Users\me\venv\lib\site-packages\twisted\conch\ssh\transport.py", line 438, in dataReceived
    self.dispatchMessage(messageNum, packet[1:])
  File "C:\Users\me\venv\lib\site-packages\twisted\conch\ssh\transport.py", line 460, in dispatchMessage
    messageNum, payload)
--- <exception caught here> ---
  File "C:\Users\me\venv\lib\site-packages\twisted\python\log.py", line 84, in callWithLogger
    return callWithContext({"system": lp}, func, *args, **kw)
  File "C:\Users\me\venv\lib\site-packages\twisted\python\log.py", line 69, in callWithContext
    return context.call({ILogContext: newCtx}, func, *args, **kw)
  File "C:\Users\me\venv\lib\site-packages\twisted\python\context.py", line 118, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "C:\Users\me\venv\lib\site-packages\twisted\python\context.py", line 81, in callWithContext
    return func(*args,**kw)
  File "C:\Users\me\venv\lib\site-packages\twisted\conch\ssh\service.py", line 44, in packetReceived
    return f(packet)
  File "C:\Users\me\venv\lib\site-packages\twisted\conch\ssh\connection.py", line 228, in ssh_CHANNEL_DATA
    channel = self.channels[localChannel]
exceptions.KeyError: 0

Looks like I'm still getting data from the server after the channel has been closed. Someone in #twisted seemed to think I shouldn't be calling serviceStopped() myself because it should be called automatically by a different part of Twisted.

I did some poking around in the Twisted source code and found that serviceStopped is supposed to be called by t.c.s.t.SSHClientTransport.connectionLost().

I'm keeping track of my SFTP client objects and accessing the SSH connection via their transport attribute. Here is an example you can run locally to demonstrate the issue. The raw can be fetched here.

from os.path import basename
import sys

from twisted.conch.client.connect import connect
from twisted.conch.client.options import ConchOptions
from twisted.internet.defer import Deferred
from twisted.conch.ssh import channel, userauth
from twisted.conch.ssh.common import NS
from twisted.conch.ssh.connection import SSHConnection
from twisted.conch.ssh.filetransfer import FXF_WRITE, FXF_CREAT, \
    FXF_TRUNC, FileTransferClient
from twisted.internet import reactor, defer
from twisted.python.log import startLogging

ACTIVE_CLIENTS = {}
USERNAME = 'user'           # change me!
PASSWORD = 'password'       # change me!
HOST = ('hostname', 22)     # change me!
TEST_FILE_PATH = __file__
TEST_FILE_NAME = basename(__file__)


def openSFTP(user, host):
    conn = SFTPConnection()
    options = ConchOptions()
    options['host'], options['port'] = host
    conn._sftp = Deferred()
    auth = SimpleUserAuth(user, conn)
    connect(options['host'], options['port'], options, verifyHostKey, auth)
    return conn._sftp


def verifyHostKey(ui, hostname, ip, key):
    return defer.succeed(True)


class SimpleUserAuth(userauth.SSHUserAuthClient):
    def getPassword(self):
        return defer.succeed(PASSWORD)


class SFTPConnection(SSHConnection):
    def serviceStarted(self):
        self.openChannel(SFTPChannel())


class SFTPChannel(channel.SSHChannel):
    name = 'session'

    def channelOpen(self, ignoredData):
        d = self.conn.sendRequest(self, 'subsystem', NS('sftp'),
                                  wantReply=True)
        d.addCallback(self._cbFTP)
        d.addErrback(self.printErr)

    def _cbFTP(self, ignore):
        client = FileTransferClient()
        client.makeConnection(self)
        self.dataReceived = client.dataReceived
        ACTIVE_CLIENTS.update({self.conn.transport.transport.addr: client})
        self.conn._sftp.callback(None)

    def printErr(self, msg):
        print msg
        return msg


@defer.inlineCallbacks
def main():
    d = openSFTP(USERNAME, HOST)
    _ = yield d

    client = ACTIVE_CLIENTS[HOST]
    d = client.openFile(TEST_FILE_NAME, FXF_WRITE | FXF_CREAT | FXF_TRUNC, {})
    df = yield d

    sf = open(TEST_FILE_PATH, 'rb')
    d = df.writeChunk(0, sf.read())
    _ = yield d

    sf.close()
    d = df.close()
    _ = yield d

    ACTIVE_CLIENTS[HOST].transport.loseConnection()
    # loseConnection() call above causes the following log messages:
    # [SSHChannel session (0) on SSHService ssh-connection on SSHClientTransport,client] sending close 0
    # [SSHChannel session (0) on SSHService ssh-connection on SSHClientTransport,client] unhandled request for exit-status
    # [SSHChannel session (0) on SSHService ssh-connection on SSHClientTransport,client] remote close
    # [SSHChannel session (0) on SSHService ssh-connection on SSHClientTransport,client] closed
    # I can see the channel closed on the server side:
    # sshd[4485]: debug1: session_exit_message: session 0 channel 0 pid 4486
    # sshd[4485]: debug1: session_exit_message: release channel 0
    # sshd[4485]: debug1: session_by_channel: session 0 channel 0

    ACTIVE_CLIENTS[HOST].transport.conn.transport.loseConnection()
    # loseConnection() call above does not close the SSH connection.

    reactor.callLater(5, reactor.stop)
    # Stopping the reactor closes the SSH connection and logs the following messages:
    # [SSHClientTransport,client] connection lost
    # [SSHClientTransport,client] Stopping factory <twisted.conch.client.direct.SSHClientFactory instance at 0x02E5AF30>
    # [-] Main loop terminated.
    # On the server side:
    # sshd[4485]: Closing connection to xxx.xxx.xxx.xxx


if __name__ == '__main__':
    startLogging(sys.stdout)
    reactor.callWhenRunning(main)
    reactor.run()

To close the SSH connection, I'm calling ACTIVE_CLIENTS[HOST].transport.conn.transport(t.c.c.d.SSHClientTransport instance).loseConnection() which calls t.c.c.d.SSHClientTransport.sendDisconnect(). Here's the sendDisconnect() method:

def sendDisconnect(self, code, reason):
    if self.factory.d is None:
        return
    d, self.factory.d = self.factory.d, None
    transport.SSHClientTransport.sendDisconnect(self, code, reason)
    d.errback(error.ConchError(reason, code))

self.factory.d seems to always be None when this method is called so it returns without calling t.c.s.t.SSHClientTransport.sendDisconnect(). I think it was originally a deferred set in t.c.c.d.connect, but at some point it is set to None.

I suspect that SSHClientTransport.loseConnection() is the correct way to close SSH connections, but why is self.factory.d set to None when twisted expects it to be something else?

If loseConnection() is not the correct way to close SSH connections could someone point me in the right direction?

Answer 1

It sounds like you're using twisted.conch.client.direct.SSHClientFactory and twisted.conch.client.direct.SSHClientTransport. These classes are most directly intended to be used to implement the conch command line tool. This means they're fairly useful for building an SSH client, since that's exactly what conch is.

However, they're also somewhat less generally useful than one might imagine, since they don't pay much attention to doing anything ''other'' than implementing the conch command line tool.

The more generally applicable SSH client transport class is twisted.conch.ssh.transport.SSHClientTransport. This class doesn't have any extra logic for implementing some particular behavior of the conch command line tool. It just has SSH client logic. For example, it doesn't have an unexplained self.factory.d check inside sendDisconnect - its sendDisconnect implementation just sends a disconnect packet and then closes the connection.