Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is the most efficient way to remove a group of indices from a list of numbers in Python 2.7?

So I was wondering how I can, using Python 2.7, most efficiently take a list of values used to represent indices like this: (but with a length of up to 250,000+)

indices = [2, 4, 5]

and remove that list of indices from a larger list like this: (3,000,000+ items)

numbers = [2, 6, 12, 20, 24, 40, 42, 51]

to get a result like this:

[2, 6, 20, 42, 51]

I'm looking for an efficient solution more than anything else. I know there are many ways to do this, however that's not my problem. Efficiency is. Also, this operation will have to be done many times and the lists will both get exponentially smaller. I do not have an equation to represent how much smaller they will get over time.

edit:

Numbers must remain sorted in a list the entire time or return to being sorted after the indices have been removed. The list called indices can either be sorted or not sorted. It doesn't even have to be in a list.

like image 634
Steven Hicken Avatar asked Nov 27 '12 00:11

Steven Hicken


2 Answers

You may want to consider using the numpy library for efficiency (which if you're dealing with lists of integers may not be a bad idea anyway):

>>> import numpy as np
>>> a = np.array([2, 6, 12, 20, 24, 40, 42, 51])
>>> np.delete(a, [2,4,5])
array([ 2,  6, 20, 42, 51])

Notes on np.delete: http://docs.scipy.org/doc/numpy/reference/generated/numpy.delete.html

It might also be worth at looking at keeping the main array as is, but maintaining a masked array (haven't done any speed tests on that either though...)

like image 78
Jon Clements Avatar answered Oct 21 '22 12:10

Jon Clements


I have a suspicion that taking whole slices between the indices might be faster than the list comprehension

def remove_indices(numbers, indices):
    result = []
    i=0
    for j in sorted(indices):
        result += numbers[i:j]
        i = j+1
    result += numbers[i:]
    return result
like image 39
John La Rooy Avatar answered Oct 21 '22 11:10

John La Rooy