Built-in magic variable names/attributes

Question

Background: For those not familiar with it, Sublime Text (and TextMate) provides syntax highlighting and other features through scopes which are defined by .tmLanguage language definition files, basically a bunch of regexes to identify various constructs in a given language, such as function definitions, various types of strings, reserved words, etc.

I'm the maintainer of the Python Improved package (available via Package Control if you're interested) that aims to be a better language definition for Python. You can read about it at GitHub if you want, but one of the key features is that it's actually maintained, unlike many of the Sublime languages that haven't been changed or updated in years.

The question: I've been focusing recently on double-underscored __magic__ stuff, and after finding this excellent treatise by Rafe Kettler on magic functions I was able to expand that part of the language definition quite a bit. However, I've had a bit less luck on finding a good list of built-in magic variable names, or magic attributes, like __class__ or __doc__. I've gone through the Data Model section of the docs, but it leaves a little bit to be desired for my purposes, and seems to focus mainly on magic method names.

So my question is, what should be included in the support.variable.magic.python scope? This is its definition so far:

\b__(all|bases|class|debug|dict|doc|file|members|metaclass|methods|module|name|slots|weakref)__\b

One of the reasons I started this project was to teach myself more about Python, and I've definitely been succeeding so far, but I'm kind of stuck at this point.

Just to be clear, I'm not looking for a favorite off-site resource (although if you have a handy link I'd appreciate it) and I'm not trying to start an opinionated discussion. All I'm trying to figure out is if this list looks reasonable as-is, or if there are any glaring errors. If you do want to be opinionated, open an issue and I'd be more than happy to discuss.

Thanks!

Answer 1

Alas, the Data Model document is the most complete thing I can think of, and it's not even really designed as an index. I'm not entirely clear on what you're looking for, though; __all__ is a module global, __slots__ is a class attribute, __weakref__ only appears as a string inside the slot list, and __module__ is a function attribute et al. I guess any special attribute that's not typically callable, then?

Of course, you can always ask Python.

>>> dir(type)
['__abstractmethods__', '__base__', '__bases__', '__basicsize__', '__call__', '__class__', '__delattr__', '__dict__', '__dictoffset__', '__dir__', '__doc__', '__eq__', '__flags__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__instancecheck__', '__itemsize__', '__le__', '__lt__', '__module__', '__mro__', '__name__', '__ne__', '__new__', '__prepare__', '__qualname__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasscheck__', '__subclasses__', '__subclasshook__', '__weakrefoffset__', 'mro']
>>> import sys
>>> dir(type(sys))
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__']

At a glance you're definitely missing __mro__ and __subclasses__. Complicating this somewhat is that there are some special methods only used by code that happens to be built into Python, rather than by the core language: examples include __format__ (used by str.format) and the various ABC methods.

I don't even know what __weakrefoffset__ is.

Note that Python 3 has a handful of new things: there's a __prepare__ method used by metaclass shenanigans, functions and methods now use magic names for their attributes rather than noise like im_self (see the "User-defined functions" section of Data Model), and there's a __qualname__ on both modules and classes.

Also, the importing PEP mentions exactly what a module loader should do, including set some magic attributes: __name__, __file__, __path__, __loader__, and __package__.

Answer 2

import gc

print("\n".join(sorted({attrname for item in gc.get_objects() for attrname in dir(item) if attrname.startswith("__")})))
#>>> __about__
#>>> __abs__
#>>> __abstractmethods__
#>>> __add__
#>>> __all__
#>>> __and__
#>>> __annotations__
#>>> __author__
#>>> __base__
#>>> __bases__
#>>> __basicsize__
#>>> __bool__
#>>> __build_class__
#>>> __builtins__
#>>> __cached__
#>>> __call__
#>>> __cause__
#>>> __ceil__
#>>> __class__
#>>> __closure__
#>>> __code__
#>>> __complex__
#>>> __concat__
#>>> __contains__
#>>> __context__
#>>> __copy__
#>>> __copyright__
#>>> __credits__
#>>> __date__
#>>> __debug__
#>>> __deepcopy__
#>>> __defaults__
#>>> __del__
#>>> __delattr__
#>>> __delete__
#>>> __delitem__
#>>> __dict__
#>>> __dictoffset__
#>>> __dir__
#>>> __displayhook__
#>>> __divmod__
#>>> __doc__
#>>> __enter__
#>>> __eq__
#>>> __excepthook__
#>>> __exit__
#>>> __file__
#>>> __flags__
#>>> __float__
#>>> __floor__
#>>> __floordiv__
#>>> __format__
#>>> __func__
#>>> __ge__
#>>> __get__
#>>> __getattr__
#>>> __getattribute__
#>>> __getitem__
#>>> __getnewargs__
#>>> __getstate__
#>>> __globals__
#>>> __gt__
#>>> __hash__
#>>> __iadd__
#>>> __iand__
#>>> __iconcat__
#>>> __ifloordiv__
#>>> __ilshift__
#>>> __imod__
#>>> __import__
#>>> __imul__
#>>> __index__
#>>> __init__
#>>> __initializing__
#>>> __instancecheck__
#>>> __int__
#>>> __inv__
#>>> __invert__
#>>> __ior__
#>>> __ipow__
#>>> __irshift__
#>>> __isabstractmethod__
#>>> __isub__
#>>> __itemsize__
#>>> __iter__
#>>> __itruediv__
#>>> __ixor__
#>>> __kwdefaults__
#>>> __le__
#>>> __len__
#>>> __loader__
#>>> __lshift__
#>>> __lt__
#>>> __missing__
#>>> __mod__
#>>> __module__
#>>> __mro__
#>>> __mul__
#>>> __name__
#>>> __ne__
#>>> __neg__
#>>> __new__
#>>> __newobj__
#>>> __next__
#>>> __not__
#>>> __objclass__
#>>> __or__
#>>> __package__
#>>> __path__
#>>> __pos__
#>>> __pow__
#>>> __prepare__
#>>> __qualname__
#>>> __radd__
#>>> __rand__
#>>> __rdivmod__
#>>> __reduce__
#>>> __reduce_ex__
#>>> __repr__
#>>> __reversed__
#>>> __rfloordiv__
#>>> __rlshift__
#>>> __rmod__
#>>> __rmul__
#>>> __ror__
#>>> __round__
#>>> __rpow__
#>>> __rrshift__
#>>> __rshift__
#>>> __rsub__
#>>> __rtruediv__
#>>> __rxor__
#>>> __self__
#>>> __set__
#>>> __setattr__
#>>> __setitem__
#>>> __setstate__
#>>> __sizeof__
#>>> __slots__
#>>> __stderr__
#>>> __stdin__
#>>> __stdout__
#>>> __str__
#>>> __sub__
#>>> __subclasscheck__
#>>> __subclasses__
#>>> __subclasshook__
#>>> __suppress_context__
#>>> __traceback__
#>>> __truediv__
#>>> __trunc__
#>>> __version__
#>>> __weakref__
#>>> __weakrefoffset__
#>>> __wrapped__
#>>> __xor__