Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python append performance

I'm having some performance problems with 'append' in Python. I'm writing an algorithm that checks if there are two overlapping circles in a (large) set of circles. I start by putting the extreme points of the circles (x_i-R_i & x_i+R_i) in a list and then sorting the list.

class Circle:
def __init__(self, middle, radius):
    self.m = middle
    self.r = radius

In between I generate N random circles and put them in the 'circles' list.

"""
Makes a list with all the extreme points of the circles.
Format = [Extreme, left/right ~ 0/1 extreme, index]
Seperate function for performance reason, python handles local variables faster.
Garbage collect is temporarily disabled since a bug in Python makes list.append run in O(n) time instead of O(1)
"""
def makeList():
    """gc.disable()"""
    list = []
    append = list.append
    for circle in circles:
        append([circle.m[0]-circle.r, 0, circles.index(circle)])
        append([circle.m[0] + circle.r, 1, circles.index(circle)])
    """gc.enable()"""
    return list

When running this with 50k circles it takes over 75 seconds to generate the list. As you might see in the comments I wrote I disabled garbage collect, put it in a separate function, used

append = list.append
append(foo)

instead of just

list.append(foo)

I disabled gc since after some searching it seems that there's a bug with python causing append to run in O(n) instead of O(c) time.

So is this way the fastest way or is there a way to make this run faster? Any help is greatly appreciated.

like image 800
Harm De Weirdt Avatar asked Apr 14 '11 12:04

Harm De Weirdt


People also ask

Is append slow in Python?

It does slow down like you claimed. (0.03 seconds for the first iteration, and 0.84 seconds for the last… quite a difference.) Obviously, if you instantiate a list but don't append it to x , it runs way faster and doesn't scale up over time.

Is append or insert faster Python?

Insert is slower when compared to append.

Is append or extend faster?

append(x) . Although, the power of this method comes in using it to place items somewhere within the list and not at the end. If you only needed to add an element to the end of the list then append works just fine for that, and is faster (which matters for large lists).

Is append constant time in Python?

Time Complexity for Append in PythonThis function has constant time complexity i.e. O(1), because lists are randomly accessed so the last element can be reached in O(1) time that's why the time taken to add the new element at the end of the list is O(1).


1 Answers

Instead of

for circle in circles:
    ... circles.index(circle) ...

use

for i, circle in enumerate(circles):
    ... i ...

This could decrease your O(n^2) to O(n).

Your whole makeList could be written as:

sum([[[circle.m[0]-circle.r, 0, i], [circle.m[0]+circle.r, 1, i]] for i, circle in enumerate(circles)], [])
like image 175
eumiro Avatar answered Sep 28 '22 07:09

eumiro