Dynamic list that automatically expands

Question

How can I make a Python equivalent of pdtolist from Pop-11?

Assume I have a generator called g that returns (say) integers one at a time. I'd like to construct a list a that grows automatically as I ask for values beyond the current end of the list. For example:

print a # => [ 0, 1, 2, g]
print a[0] # => 0
print a[1] # => 1
print a[2] # => 2
# (obvious enough up to here)

print a[6] # => 6
print a # => [ 0, 1, 2, 3, 4, 5, 6, g]
# list has automatically expanded

a = a[4:] # discard some previous values
print a # => [ 4, 5, 6, g]
print a[0] # => 4

Terminology - to anticipate a likely misunderstanding: a list is a "dynamic array" but that's not what I mean; I'd like a "dynamic list" in a more abstract sense.

To explain the motivation better, suppose you have 999999999 items to process. Trying to fit all those into memory (in a normal list) all at once would be a challenge. A generator solves that part of the problem by presenting them one at a time; each one created on demand or read individually from disk. But suppose during processing you want to refer to some recent values, not just the current one? You could remember the last (say) ten values in a separate list. But a dynamic list is better, as it remembers them automatically.

Answer 1

This might get you started:

class DynamicList(list):
    def __init__(self, gen):
        self._gen = gen

    def __getitem__(self, index):
        while index >= len(self):
            self.append(next(self._gen))
        return super(DynamicList, self).__getitem__(index)

You'll need to add some special handling for slices (currently, they just return a normal list, so you lose the dynamic behavior). Also, if you want the generator itself to be a list item, that'll add a bit of complexity.

Answer 2

Just answered another similar question and decided to update my answer for you hows this?

class dynamic_list(list):
    def __init__(self,num_gen):
        self._num_gen = num_gen
    def __getitem__(self,index):
        if isinstance(index, int):
            self.expandfor(index)
            return super(dynamic_list,self).__getitem__(index)

        elif isinstance(index, slice):
            if index.stop<index.start:
                return super(dynamic_list,self).__getitem__(index)
            else:
                self.expandfor(index.stop if abs(index.stop)>abs(index.start) else index.start)
            return super(dynamic_list,self).__getitem__(index)

    def __setitem__(self,index,value):
        if isinstance(index, int):
            self.expandfor(index)
            return super(dynamic_list,self).__setitem__(index,value)

        elif isinstance(index, slice):
            if index.stop<index.start:
                return super(dynamic_list,self).__setitem__(index,value)
            else:
                self.expandfor(index.stop if abs(index.stop)>abs(index.start) else index.start)
            return super(dynamic_list,self).__setitem__(index,value)

    def expandfor(self,index):
            rng = []
            if abs(index)>len(self)-1:
                if index<0:
                    rng = xrange(abs(index)-len(self))
                else:
                    rng = xrange(abs(index)-len(self)+1)
            for i in rng:
                self.append(self._num_gen.next())