Private attributes in Python and PEP8

Question

I was just re-reading through PEP8 and stumbled onto this line:

We don't use the term "private" here, since no attribute is really private in Python (without a generally unnecessary amount of work).

I was just wondering what would this "unnecessary amount of work" entail if I wanted to make a method truly private?

Answer 1

In pure python, the closest you can get to "true" encapsulation is via variables in closure (since you cannot access/assign them directly once a function has returned). Here's a small sketch:

def OneTruePrivate():
    """hax which uses a closure to make _truly_ private variables"""
    x = 1

    class OneTruePrivate:
        @property
        def val(self):
            return x

        def increment(self):
            nonlocal x
            x += 1

    return OneTruePrivate()


var = OneTruePrivate()
print('initial value')
print(var.val)
var.increment()
print('after incremnet')
print(var.val)
print('look ma, no object state!')
import pprint; pprint.pprint(vars(var))

output:

initial value
1
after incremnet
2
look ma, no object state!
{}

that said, even with some hackery you can mutate the underlying closure value

cell = var.increment.__func__.__closure__[0]
import ctypes
ctypes.pythonapi.PyCell_Set.argtypes = (ctypes.py_object, ctypes.py_object)
ctypes.pythonapi.PyCell_Set(cell, 9001)
print(var.val)

output:

9001

Answer 2

There's a spectrum for this. More privacy takes more work, but is harder to bypass. There's no clear line for "really private".

Leading double underscores are the least private option. It's easy to stick leading double underscores on an attribute, but also easy to bypass by manual name mangling.

Code and bypass:

class Private1:
    def __init__(self, data):
        self.__data = data
    def get_data(self):
        return self.__data

x = Private1(1)

# Direct access fails...
x.__data = 2

# but you can do the name mangling manually.
x._Private1__data = 2

The next up would be stuff like closure variables or hidden slots. You can't access those by name mangling, but you can still manually access closure cells or figure out where the slot getter is.

Closure variable example, with bypass:

class Private2:
    def __init__(self, data):
        def get_data():
            return data
        self.get_data = get_data

x = Private2(1)

# It's not immediately obvious how you'd even try to access the data variable directly,
# but you can:
x.get_data.__closure__[0].cell_contents = 2

Hidden slot example, with bypass:

class Private3:
    __slots__ = ('data',)
    def __init__(self, data):
        _hidden_slot.__set__(self, data)
    def get_data(self):
        return _hidden_slot.__get__(self, type(self))
_hidden_slot = Private3.data
del Private3.data

x = Private3(1)

# Direct access fails...
x.data = 2

# but you can directly access the slot getter the same way the actual class did:
_hidden_slot.__set__(x, 2)

The next step up after that would be C extensions. Writing a C extension manually is a lot of work, enough that I'm not going to bother with an example (but here's a tutorial link, and Cython makes it a lot easier), but types implemented in C don't expose their internal data at Python level by default. If the type doesn't make a specific effort to provide access, the only way to access the data would be with more C, or with stuff like ctypes, or gc.get_referents if the hidden data is a GC-exposed Python reference. (The right use of gc.get_referents can also bypass all the other above protections.)

The next step up after that would be to hold the data on your own private server, and only let clients access it through an internet API. That's much more private than any private keyword, and bypassing that requires something like a vulnerability exploit or a subpoena or physical violence or something.