Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python: Making a class variable static even when a module is imported in different ways

Let's consider the package structure as below:

myApp
|-- myPackage
|    |-- __init__.py 
|    +-- myModule.py
|-- __init__.py
+-- main.py

myModule.py contains a single class such as:

class MyClass( object ):
    _myList = []

    @classmethod
    def test( cls ):
        cls._myList.append( len( cls._myList ) )
        return cls._myList

    def __init__( self ):
        return

    pass

As you can see, there's nothing fancy, I'm just defining a list as a static class variable. Now let's consider the code from main.py:

from myApp.myPackage.myModule import MyClass as MyClassAbs
from myPackage.myModule import MyClass as MyClassRel

if __name__ == '__main__':

    print '\n  myList'
    print 'MyClassAbs().test():', MyClassAbs().test() #< Prints [0].
    print 'MyClassRel().test():', MyClassRel().test() #< Prints [0] but should be [0, 1].
    print 'MyClassAbs.test():', MyClassAbs.test() #< Prints [0, 1] but should be [0, 1, 2].
    print 'MyClassRel.test():', MyClassRel.test() #< Prints [0, 1] but should be [0, 1, 2, 3].

    print '\n  myList ids'
    print id( MyClassAbs().test() )
    print id( MyClassRel().test() )
    print id( MyClassAbs.test() )
    print id( MyClassRel.test() )

    print ''
    print 'MyClassAbs == MyClassRel:', MyClassAbs == MyClassRel #< Prints False
    print 'isinstance( MyClassAbs(), MyClassRel ):', isinstance( MyClassAbs(), MyClassRel ) #< Prints False
    print 'isinstance( MyClassRel(), MyClassAbs ):', isinstance( MyClassRel(), MyClassAbs ) #< Prints False

    pass

The issue is that in my code I'm importing twice the same class from the same module but in different ways: once as a an absolute import and once as a relative one. As seen in the last part of the code, even though the classes are the same, they are not equal because their module are being registered distinctively into the sys.modules:

'myApp.myPackage.myModule': <module 'myApp.myPackage.myModule' from '/full/path/myApp/myPackage/myModule.py'>
'myPackage.myModule': <module 'myPackage.myModule' from '/full/path/myApp/myPackage/myModule.py'>

and as a result, their representation differs:

'MyClassAbs': <class 'myApp.myPackage.myModule.MyClass'>
'MyClassRel': <class 'myPackage.myModule.MyClass'>

So... is there any way to set this variable as static for good?

Edit:
The code above is a obviously only a reduction of my real problem. In reality, I basically have a piece of code that inspects all the modules recursively nested within a folder and that registers the classes contained in there. All the classes registered this way can then be instanciated using a common syntax such as

myApp.create( 'myClass' )

This is why I sometimes end up having 2 objects pointing to the same class but that have been imported in different ways. One has been imported automatically through a imp.load_module() call, and the other would have been directly imported by the user through a conventional import statement. If the user decides to manually import a class, he should still have access to the same static class variable than the one defined within the same but automatically registered class. Hope it makes sense?

like image 647
ChristopherC Avatar asked Nov 15 '12 05:11

ChristopherC


1 Answers

No. (At least, not without very ugly and fragile hacks.) When you import it in those two different ways, the module is actually imported twice. Python usually re-uses an already-imported module if you import it again, but this is based not on the actual file imported, but on the path relative to sys.path. So if you import myModule.py once absolutely and once relatively, the whole file actually gets executed twice. You can see this if you do something like:

from myApp.myPackage import myModule
import myApp.myPackage.myModule as sameModule
import myPackage.myModule
print myModule is sameModule
print myModule is myPackage.myModule

You should see True for the first print (because the first two imports are via the same path) but False for the second (because the third import uses a different path). This is also why you're seeing the module with two entries in sys.modules.

Because the whole module is imported twice, you actually have two different classes called MyClass, not one. There's no legitimate way to have those two classes share their state. It's just as if you'd imported two different classes from two different modules. Even though they happen to be defined in the same source file, they're not linked in any way when you double-import them like that. (There is an evil way to link them, which is that you can include code in your module to manually check if it's been imported under a different name, and then manually mess with sys.modules to set the currently-importing module to the already-imported one. But this is a bad idea, because it can be hard to tell if what's being imported is really the same module, and also because stomping on sys.modules can lead to strange bugs if either the original import or the new import failed for any reason.)

The real question is, why are you importing the module twice? The second question is, why are you using relative imports at all? The solution is to just import the module once using absolute imports, and then you'll have no problems.

like image 114
BrenBarn Avatar answered Sep 17 '22 01:09

BrenBarn