How can I create an indexable map(), or a decorated list()?

Question

I have a long list of file paths, like:

images = ['path/to/1.png', 'path/to/2.png']

I know that I can wrap this list in a map iterator, which provides sequential access to items in list mapped through a function, like:

image_map = map(cv2.imread, images)

Then I can lazily load those image files as I iterate over the list:

next(image_map)
=> pixels

But I want random access to the original list, mapped through my map function:

image_map[400]
=> pixels

I don't want to convert it to a list, because that would load all of my images into memory, and they don't fit into memory:

# Bad:
list(image_map)[400]

Another way to think about this might be a decorator on list.__getitem__.

I know that I can sub-class a list, but I'm really hoping for a cleaner way of doing this.

Answer 1

Why don't you just create an accessor class?

class ImageList(object):
    def __init__(self, images):
        self.images = images

    def get_image(self, image_num):
        return cv2.imread(self.images[image_num])

You of course also can buffer read images.

You can also provide a __getitem__ method to have list-like access:

def __getitem__(self, key):
    return cv2.imread(self.images[key])

Usage:

images = ['path/to/1.png', 'path/to/2.png']

image_list = ImageList(images)

image = image_list.get_image(400)    # the same as 
image = image_list[400]              # this

By the way: Of course you could subclass list but in the Python community more explicite is favored. And it is simply more clear here to have a separate class instead of sub-classing list. It is also not the best style to over-use inheritance.

Answer 2

One issue with using map if you can't convert it the result to a list is that you can only iterate through it once. If you can't fit the whole thing into memory at once and you need random access, then you're going to want to keep around as little information as possible until you really need it -- which sounds like when __getitem__ is called. Thus, if you're intent on being able to write image_map[n] to get the pixel data (instead of just calling cv2.imread(image_map[n]), you will have to create a list subclass whose __getitem__ calls cv2.imread.

e.g.

class cv2_list(list):
    def __getitem__(self, item):
        return cv2.imread(super().__getitem__(item))