What is the purpose of deepcopy's second parameter, memo?

Question

from copy import*  a=[1,2,3,4] c={'a':'aaa'} print c #{'a': 'aaa'} b=deepcopy(a,c) print b  print c # print {'a': 'aaa', 10310992: 3, 10310980: 4, 10311016: 1, 11588784: [1, 2, 3, 4, [1, 2, 3, 4]], 11566456: [1, 2, 3, 4], 10311004: 2} 

why c print that

Please try to use the code, rather than text, because my English is not very good, thank you

in django.utils.tree.py

def __deepcopy__(self, memodict):         """         Utility method used by copy.deepcopy().         """         obj = Node(connector=self.connector, negated=self.negated)         obj.__class__ = self.__class__         obj.children = deepcopy(self.children, memodict)         obj.subtree_parents = deepcopy(self.subtree_parents, memodict)         return obj    import copy memo = {} x1 = range(5) x2=range(6,9) x3=[2,3,4,11] y1 = copy.deepcopy(x1, memo) y2=copy.deepcopy(x2, memo) y3=copy.deepcopy(x3,memo) print memo print id(y1),id(y2),id(y3) y1[0]='www' print y1,y2,y3 print memo 

print :

{10310992: 3, 10310980: 4, 10311016: 1, 11588784: [0, 1, 2, 3, 4, [0, 1, 2, 3, 4]], 10311028: 0, 11566456: [0, 1, 2, 3, 4], 10311004: 2} {11572448: [6, 7, 8], 10310992: 3, 10310980: 4, 10311016: 1, 11572368: [2, 3, 4, 11], 10310956: 6, 10310896: 11, 10310944: 7, 11588784: [0, 1, 2, 3, 4, [0, 1, 2, 3, 4], 6, 7, 8, [6, 7, 8], 11, [2, 3, 4, 11]], 10311028: 0, 11566456: [0, 1, 2, 3, 4], 10310932: 8, 10311004: 2} 11572408 11581280 11580960 ['www', 1, 2, 3, 4] [6, 7, 8] [2, 3, 4, 11] {11572448: [6, 7, 8], 10310992: 3, 10310980: 4, 10311016: 1, 11572368: [2, 3, 4, 11], 10310956: 6, 10310896: 11, 10310944: 7, 11588784: [0, 1, 2, 3, 4, [0, 1, 2, 3, 4], 6, 7, 8, [6, 7, 8], 11, [2, 3, 4, 11]], 10311028: 0, 11566456: ['www', 1, 2, 3, 4], 10310932: 8, 10311004: 2} 

Answer 1

It's the memo dict, where id-to-object correspondence is kept to reconstruct complex object graphs perfectly. Hard to "use the code", but, let's try:

>>> import copy >>> memo = {} >>> x = range(5) >>> y = copy.deepcopy(x, memo) >>> memo {399680: [0, 1, 2, 3, 4], 16790896: 3, 16790884: 4, 16790920: 1,  438608: [0, 1, 2, 3, 4, [0, 1, 2, 3, 4]], 16790932: 0, 16790908: 2} >>>  

and

>>> id(x) 399680 >>> for j in x: print j, id(j) ...  0 16790932 1 16790920 2 16790908 3 16790896 4 16790884 

so as you see the IDs are exactly right. Also:

>>> for k, v in memo.items(): print k, id(v) ...  399680 435264 16790896 16790896 16790884 16790884 16790920 16790920 438608 435464 16790932 16790932 16790908 16790908 

you see the identity for the (immutable) integers.

So here's a graph:

>>> z = [x, x] >>> t = copy.deepcopy(z, memo) >>> print id(t[0]), id(t[1]), id(y) 435264 435264 435264 

so you see all the subcopies are the same objects as y (since we reused the memo).

Answer 2

No one above gave a good example of how to use it.

Here's what I do:

def __deepcopy__(self, memo):     copy = type(self)()     memo[id(self)] = copy     copy._member1 = self._member1     copy._member2 = deepcopy(self._member2, memo)     return copy 

Where member1 is an object not requiring deepcopy (like a string or integer), and member2 is one that does, like another custom type or a list or dict.

I've used the above code on highly tangled object graphs and it works very well.

If you also want to make your classes pickleable (for file save / load), there is not analogous memo param for getstate / setstate, in other words the pickle system somehow keeps track of already referenced objects, so you don't need to worry.

The above works on PyQt5 classes that you inherit from (as well as pickling - for instance I can deepcopy or pickle a custom QMainWindow, QWidget, QGraphicsItem, etc.)

If there is some initialization code in your constructor that creates new objects, for instance a CustomWidget(QWidget) that creates a new CustomScene(QGraphicsScene), but you'd like to pickle or copy the scene from one CustomWidget to a new one, then one way is to make a new=True parameter in your __init__ and say:

def __init__(..., new=True):     ....     if new:        self._scene = CustomScene()  def __deepcopy__(self, memo):     copy = type(self)(..., new=False)     ....     copy._scene = deepcopy(self._scene, memo)     .... 

That ensures you don't create a CustomScene (or some big class that does a lot of initializing) twice! You also should use the same setting (new=False) in your __setstate__ method, eg.:

 def __setstate__(self, data):      self.__init__(...., new=False)      self._member1 = data['member 1']      ..... 

There are other ways to get around the above, but this is the one I converged to and use frequently.

Why did I talk about pickling as well? Because you will want both in any application typically, and you maintain them at the same time. If you add a member to your class, you add it to setstate, getstate, and deepcopy code. I would make it a rule that for any new class you make, you create the above three methods if you plan on doing copy / paste an file save / load in your app. Alternative is JSON and save / loading yourself, but then there's a lot more work for you to do including memoization.

So to support all the above, you need __deepcopy__, __setstate__, and __getstate__ methods and to import deepcopy:

from copy import deepcopy 

, and when you write your pickle loader / saver functions (where you call pickle.load()/ pickle.dump() to load / save your object hierarchy / graph) do import _pickle as pickle for the best speeds (_pickle is some faster C impl which is usually compatible with your app requirements).