Python Class: overwrite `self`

Question

In my python script I have a global storage (a simple global dict) that stores Processo objects. It get's filled during the execution of my program. It exists to avoid creating repeated Processo objects, due performance reasons.

So, for the class Processo I want verify during its creation if it is already on the global storage.

In that case I just want to copy it to self. I am using getfromStorage() for that.

class Processo:
    def __init__(self, name, ...): # ... for simplicity
       self.processoname = name
       self = getfromStorage(self)

Don't know if it's useful but ...

def getfromStorage(processo):
    if processo.processoname in process_storage:
        return process_storage[processo.processoname]
    return processo

How do I achieve that? Am I missing something or my design is wrong?

Answer 1

This pattern can't be accomplished reasonably with __init__, because __init__ only initializes an already existing object, and you can't change what the caller will get (you can rebind self, but that just cuts you off from the object being created, the caller has their own separate alias that is unaffected).

The correct way to do this is to override the actual constructor, __new__, which allows you to return the new instance, which you may or may not create:

class Processo:
    def __new__(cls, name, ...): # ... for simplicity
       try:
           # Try to return existing instance from storage
           return getfromStorage(name)
       except KeyError:
           pass

       # No instance existed, so create new object
       self = super().__new__(cls)  # Calls parent __new__ to make empty object

       # Assign attributes as normal
       self.processoname = name

       # Optionally insert into storage here, e.g. with:
       self = process_storage.setdefault(name, self)
       # which will (at least for name of built-in type) atomically get either then newly
       # constructed self, or an instance that was inserted by another thread
       # between your original test and now
       # If you're not on CPython, or name is a user-defined type where __hash__
       # is implemented in Python and could allow the GIL to swap, then use a lock
       # around this line, e.g. with process_storage_lock: to guarantee no races

       # Return newly constructed object
       return self

To reduce overhead, I mildly rewrote getfromStorage, so it just takes the name and performs lookup, allowing the exception to bubble if it fails:

def getfromStorage(processoname):
    return process_storage[processoname]

which means that, when a cached instance can be used, no unnecessary self object need be freshly constructed.

Note: If you do this, it's usually a good idea not to define __init__ at all; the construction of an object is done by calling the class's __new__, then implicitly calling __init__ on the result. For cached instances, you wouldn't want them reinitialized, so you want an empty __init__ (so the cached instance isn't modified by virtue of being retrieved from the cache). Put all __init__-like behavior in the code that constructs and returns a new object inside __new__, and only execute it for new objects, to avoid this problem.

Answer 2

The first question I would ask is why method getfromStorage is not exposed to the user with the implementation:

def getfromStorage(name):
    if name in process_storage:
        return process_storage[name]
    return Processo(name)

And Processo defined as:

class Processo:
    def __init__(self, name, ...): # ... for simplicity
       self.processoname = name

This seems like it would solve your problem. But if you do not want to or cannot expose the underlying caching, then:

Another Way Similar To What You Are Trying To Do

I believe the pattern that fits your need is the so-called Bridge pattern, also known as the Handle/Body pattern. Quoting from Design Patterns by Gamma et al, this pattern is used in a variety of circumstances among which is when "you want to share an implementation among multiple objects and this fact should be hidden from the client or you always want the client to access the actual "body" implementation through the "handle" object for some other reason. The "handle" object keeps a reference to the "body" implementation and delegates all calls to that implementation. Multiple handles can refer to the same body, of course. In the example below, I have used an abstract base class, which really isn't necessary with Python's duck typing.

from abc import ABC, abstractmethod

class Processo(ABC):

    @abstractmethod
    def __init__(self, name): raise NotImplementedError

    @abstractmethod
    def get_name(self): raise NotImplementedError

    @abstractmethod
    def foo(self): raise NotImplementedError


class Processo_Handle(Processo): # the handle class
    def __init__(self, name):
        self.processo = getfromStorage(name)

    def get_name(self):
        return self.processo.get_name() # delegate to "body" object

    def foo(self):
        self.processo.foo() # delegate to "body" object
        return self # don't return result of the call to foo because it's the wrong instance (i.e. the body)


class Processo_Body(Processo): # the bodyclass
    def __init__(self, name):
        self.name = name

    def get_name(self):
        return self.name

    def foo(self):
        """ do whatever it takes """
        return self

process_storage = {} # dummy implementation    
def getfromStorage(processoname):
    if processoname in process_storage:
        return process_storage[processoname]
    return Processo_Body(processoname)