What kind of python magic does dir() perform with __getattr__?

Question

The following is in python 2.7 with MySQLdb 1.2.3.

I needed a class wrapper to add some attributes to objects which didn't support it (classes with __slots__ and/or some class written in C) so I came out with something like this:

class Wrapper(object):

    def __init__(self, obj):
        self._wrapped_obj = obj

    def __getattr__(self, obj):
        return getattr(self._wrapped_obj, attr)

I was expecting that the dir() builtin called on my instance of Wrapper should have returned just the names inherited by object plus wrapped_obj, and I discovered that this is actually the case for most cases, but not for all. I tried this with a custom old style class, a custom new style class, and some builtin classes, it always worked this way: the only exception that i found is when the wrapped object was an instance of the class _mysql.connection. In this case, dir() on my object happens to know also all the method names attached to the wrapped connection object.

I read in the python documentation about dir, and this behaviour appears to be legit: dir is supposed to return a list of "interesting names", not the "real" content of the instance. But I really can't figure how it does this: it actually understands the implementation of my __getattr__ and resolves to the attached item? If this is true, why only with that connection class and not for instance with a simpler dict?

Here is some pasted code as an example of this curious behaviour:

>>> from _mysql import connection
>>> c = connection(**connection_parameters)
>>> c
<_mysql.connection open to '127.0.0.1' at a16920>
>>> 
>>> dir(c)
['affected_rows', 'autocommit', 'change_user', 'character_set_name', 'close', 'commit', 'dump_debug_info', 'errno', 'error', 'escape', 'escape_string', 'field_count', 'get_character_set_info', 'get_host_info', 'get_proto_info', 'get_server_info', 'info', 'insert_id', 'kill', 'next_result', 'ping', 'query', 'rollback', 'select_db', 'set_character_set', 'set_server_option', 'shutdown', 'sqlstate', 'stat', 'store_result', 'string_literal', 'thread_id', 'use_result', 'warning_count']
>>> 
>>> w = Wrapper(c)
>>> dir(w)
['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattr__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_wrapped_obj', 'affected_rows', 'autocommit', 'change_user', 'character_set_name', 'close', 'commit', 'dump_debug_info', 'errno', 'error', 'escape', 'escape_string', 'field_count', 'get_character_set_info', 'get_host_info', 'get_proto_info', 'get_server_info', 'info', 'insert_id', 'kill', 'next_result', 'ping', 'query', 'rollback', 'select_db', 'set_character_set', 'set_server_option', 'shutdown', 'sqlstate', 'stat', 'store_result', 'string_literal', 'thread_id', 'use_result', 'warning_count']
>>> 
>>> d = Wrapper({})
>>> dir(d)
['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattr__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_wrapped_obj']
>>> 

Answer 1

There are two deprecated attributes in Python 2, object.__members__ and object.__methods__; these were aimed at supporting dir() on extension types (C-defined objects):

object.__methods__
Deprecated since version 2.2: Use the built-in function dir() to get a list of an object’s attributes. This attribute is no longer available.

object.__members__
Deprecated since version 2.2: Use the built-in function dir() to get a list of an object’s attributes. This attribute is no longer available.

These were removed from Python 3, but because your connection object (at leasts in the older version you are using) still provides a __methods__ attribute that is found through your __getattr__ hook and used by dir() here.

If you add a print statement to the __getattr__ method you'll see the attributes being accessed:

>>> class Wrapper(object):
...     def __init__(self, obj):
...         self._wrapped_obj = obj
...     def __getattr__(self, obj):
...         print 'getattr', obj
...         return getattr(self._wrapped_obj, attr)
... 
>>> dir(Wrapper({}))
getattr __members__
getattr __methods__
['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattr__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_wrapped_obj']

For new-style objects, the newer __dir__ method supported by dir() is properly looked up on the type only so you don't see that being accessed here.

The project HISTORY file suggests the attributes were removed in the big Python 3 compatibility update for 1.2.4 beta 1.