Avoid deepcopy due to performance

Question

I have a list with another 3 lists in it. I need to do some operations on the lists, as i have like 180.000 of those the step of deepcopying already takes 2,5s to copy the lists once. The overall time including operations takes 80s out of 220s compution time.

s = [[1,2,3],
     [4,5,6],
     [7,8,9]]
s1 = copy.deepcopy(s)
s1[0][0] = s[1][1]
s1[1][1] = s[0][0]

The shown operation needs to be repeated like a million of times. So deepcopy makes me face a performance bottleneck.

Is there a more performant way or a different approach to "unreference" the list s?

Answer 1

deepcopy seems to have some overhead for checking all those different cases it is able to handle. If your lists are always lists of lists (one level of nesting), then you can try to use just s = [list(x) for x in s] or s = list(map(list, s)). Both seem to be quite a bit faster:

In [9]: s = [[random.randint(1, 1000) for _ in range(100)] for _ in range(100)]
In [10]: %timeit copy.deepcopy(s)
10 loops, best of 3: 22.7 ms per loop
In [11]: %timeit [list(x) for x in s]
10000 loops, best of 3: 123 µs per loop
In [18]: %timeit list(map(list, s))
10000 loops, best of 3: 111 µs per loop

---

Alternatively, depending on your application, it might be better not to copy and store the (modified) lists themselves, but just the modification, either in the form of the modified cells, or as a command stack.