twisted manhole: how to access servers in application?

Question

I need to connect to my twisted application at runtime and I am trying to get twisted.manhole to work for me to that end. I am on Mac OSX 10.6 with twisted 8.2 as installed by default.

The sample server using twistd works. There are DeprecationWarnings at startup about md5, sha and twisted.protocols.telnet, but the manhole server actually does what it is supposed to and I can access internals of my application:

host:client user$ telnet localhost 4040
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

twisted.manhole.telnet.ShellFactory
Twisted 8.2.0
username: admin
password: *****
>>> dir()
['_', '__builtins__', 'factory', 'service']
>>> factory
<twisted.manhole.telnet.ShellFactory instance at 0x101256440>
>>> service
<twisted.application.internet.TCPServer instance at 0x10124ff38>
>>> service.parent
<twisted.application.service.MultiService instance at 0x1014b0cf8>
>>> 

Now I try to integrate this into my application:

# test_manhole.tac

from twisted.application.internet import TCPServer
from twisted.application.service  import Application, IServiceCollection
from twisted.manhole.telnet       import ShellFactory

shell_factory = ShellFactory()
shell_factory.username = 'admin'
shell_factory.password = 'admin'
shell_factory.namespace['some_value'] = 42
shell_tcp_server = TCPServer(4040, shell_factory)

application = Application('test')
serviceCollection = IServiceCollection(application)

shell_tcp_server.setServiceParent(serviceCollection)

Run the above code in a shell:

host:server user$ twistd -noy test_manhole.tac
(omitting the same DeprecationWarnings about md5, sha and twisted.protocols.telnet as earlier)
2011-08-24 16:52:13+1000 [-] Log opened.
2011-08-24 16:52:13+1000 [-] twistd 8.2.0 (/usr/bin/python2.6 2.6.1) starting up.
2011-08-24 16:52:13+1000 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2011-08-24 16:52:13+1000 [-] twisted.manhole.telnet.ShellFactory starting on 4040
2011-08-24 16:52:13+1000 [-] Starting factory <twisted.manhole.telnet.ShellFactory instance at 0x1012cfdd0>
2011-08-24 16:52:13+1000 [-] start service:  <twisted.application.internet.TCPServer instance at 0x1012cff80>

In a second shell, run a telnet client:

host:client user$ telnet localhost 4040
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

twisted.manhole.telnet.ShellFactory
Twisted 8.2.0
username: admin
password: *****
>>> dir()
['_', '__builtins__', 'factory', 'service', 'some_value']
>>> factory
<twisted.manhole.telnet.ShellFactory instance at 0x1012cfdd0>
>>> service
>>> service == None
True
>>> service.parent
      ...
exceptions.AttributeError: 'NoneType' object has no attribute 'parent'
>>> some_value
42

So it seems the service object is not usable to access application internals.

OK, since twisted.protocols.telnet anyway seems to have been superseded by twisted.conch.telnet, try using the newer module:

# test_manhole_2.tac

from twisted.application.service import Application, IServiceCollection
from twisted.conch.manhole_tap   import makeService

options = \
{
    # for some reason, these must
    # all exist, even if None
    'namespace'  : None,
    'passwd'     : 'users.txt',
    'sshPort'    : None,
    'telnetPort' : '4040',
}

shell_service = makeService(options)

application = Application('test')
serviceCollection = IServiceCollection(application)

shell_service.setServiceParent(serviceCollection)

The 'password file' users.txt may contain as little as one line with a sample username and password, e.g. admin:admin.

Start the test server:

host:server user$ twistd -noy test_manhole_2.tac
(omitting DeprecationWarnings about md5 and sha)
2011-08-24 17:44:26+1000 [-] Log opened.
2011-08-24 17:44:26+1000 [-] twistd 8.2.0 (/usr/bin/python2.6 2.6.1) starting up.
2011-08-24 17:44:26+1000 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2011-08-24 17:44:26+1000 [-] twisted.internet.protocol.ServerFactory starting on 4040
2011-08-24 17:44:26+1000 [-] Starting factory <twisted.internet.protocol.ServerFactory instance at 0x10181e710>
2011-08-24 17:44:26+1000 [-] start service:  <twisted.application.service.MultiService instance at 0x1012553b0>
2011-08-24 17:44:26+1000 [-] start service:  <twisted.application.internet.TCPServer instance at 0x10181e998>

In a second shell, run a telnet client - note that some of this is actually guesswork as the buggy Mac OSX readline (or whatever else is the to blame) seems to swallow some shell output:

host:client user$ telnet localhost 4040
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Username: admin
Password: *****

>>> dir()
['__builtins__']

So now it seems I've gone from having a useless service object to no object at all.

How is twisted.manhole used correctly ?

Answer 1

Keep using twisted.conch and put something in the namespace dictionary.

namespace = {"your_application_object": some_object}

options = {
    # for some reason, these must
    # all exist, even if None
    'namespace'  : namespace,
    'passwd'     : 'users.txt',
    'sshPort'    : None,
    'telnetPort' : '4040',
}