How to implement a growing list

Question

I am attempting to iterate through the same data, but want to only load the data if it is necessary. I also want to reuse the fetched data in multiple, concurrent, iterators.

Is there an object, or a pattern, that would allow me to create an underlying data that has an opportunity to fetch more data if it runs out, but uses the same fetched data across multiple instances? The list should be able to be used concurrently. I happen to be using Java.

I have currently created an Iterator that returns values that can be used, if it runs out of data in its buffer, it goes and looks up more data to be returned. Unfortunately, this means that if there are multiple instances of the object, it will process the data once for each instance.

If I use a LinkedList I am concerned about ConcurrentModificationError.

UPDATE: The answer below does discuss a valid means to implement this, but I have had a lot of problems caused by the concept in general (non-standard, confuses implementors, benefits not worth the confusion). While I continue to believe this is an interest and valuable avenue, it can be problematic. As ever, before asking "how" to do something, you should ask "if" you should do it at all.

Answer 1

Here's how I'd do it.

• Create a class that implements Iterable.
• The class needs a private variable of type ArrayList which is initialized to an empty list.
• The class needs a fill method that fetches one or more entries from the source of data and adds them to the end of the private list.
• The class needs a noMore flag that is set when the fill method can't get any more data.
• The class needs an iterator() method to deliver new instances of an inner Iterator class:
  • Each iterator needs a private index ... and position in the parent classes list.
  • The hasNext method tests the index against the parent classes list size(). If they are the same it calls fill. If the noMore flag is set, it returns false.
  • The next method ... well you get the idea.

All of this needs to be properly synchronized ...

---

This approach avoids the problem of ConcurrentModificationExceptions by hiding the private list, and not using an Iterator on it. The actual iterators would use get(int) on the private list to fetch elements.

Note: this approach won't work predictably if the private list is exposed allowing other code to update or iterate the list in the normal way.