Itemgetter Except Columns

Question

Is there a way to get the complement of a set of columns using itemgetter?

For example, you can get the first, third, and fifth elements of a list using

from operator import itemgetter
f = itemgetter(0, 2, 4)
f(['a', 'b', 'c', 'd', 'e']) ## == ('a', 'c', 'e')

Is there a (simple and performant) way to get all of the elements except for the first, third and fifth?

Answer 1

No, there is no way to spell everything but these indices in Python.

You'd have to lock down the length of all inputs and hardcode the included indices, so itemgetter(*(i for i in range(fixed_list_length) if i not in {0, 2, 4})), but then you'd be locked down to processing only objects of a specific length.

If your inputs are of variable length, then one distant option is to use slices to get everything after the 4th element:

itemgetter(1, 3, slice(5, None))

but then you'd get a separate list for the slice component:

>>> itemgetter(1, 3, slice(5, None))(['a', 'b', 'c', 'd', 'e', 'f', 'g'])
('b', 'd', ['f', 'g'])

and an error if the input sequence is not at least 4 elements long:

>>> itemgetter(1, 3, slice(5, None))(['a', 'b', 'c'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list index out of range

Rather than use itemgetter(), just use a set and a lambda that uses a list comprehension:

def excludedgetter(*indices):
    excluded = set(indices)
    return lambda seq: [v for i, v in enumerate(seq) if i not in excluded]

That callable can be used for inputs of any length:

>>> from random import randrange
>>> pile = [
...     [randrange(10) for _ in range(randrange(8))]
...     for _ in range(10)
... ]
>>> min(len(l) for l in pile), max(len(l) for l in pile)
(0, 6)
>>> sorted(pile, key=excludedgetter(0, 2, 4))
[[], [1], [9, 1, 8, 2, 4, 0], [0, 3], [7, 3, 4, 9, 7, 7], [8, 4, 4], [6, 4, 7, 9, 9], [0, 5, 3, 7, 2], [4, 6, 6, 0], [8, 8, 1]]

Those random-length lists are no problem.

Answer 2

Since you're asking about itemgetter() specifically: you could use a set to get the difference:

>>> from operator import itemgetter

>>> obj = ['a', 'b', 'c', 'd', 'e']
>>> c = {1, 3, 5}  # Get everything but these
>>> get = set(range(len(obj))).difference(c)
>>> f = itemgetter(*get)
>>> f(obj)
('a', 'c', 'e')

where set(range(len(obj))) is all the indices, i.e. {0, 1, 2, 3, 4}.

---

Disclaimer: this will not guarantee sortedness given that sets are unordered. While it is a bit less efficient, you could be safer with:

f = itemgetter(*sorted(get))

---

Granted, this requires you to know the length of the list in advance, prior to the call to itemgetter(), and requires a call to that function for indexing each list.

Answer 3

You're looking for a quasi-vectorised operation. This isn't possible with regular Python, or even with 3rd party NumPy where the result is an array. But the latter does offer syntactic benefits:

import numpy as np

A = ['a', 'b', 'c', 'd', 'e']

exc = [0, 2, 4]

res1 = [val for idx, val in enumerate(A) if idx not in exc]
res2 = np.delete(A, exc).tolist()

assert res1 == res2

If you use the list comprehension, you should covert exc to set first to enable O(1) lookup.