How could I make my own lazy iterator?

Question

I'm making a C++11 class that produces a huge amount of data. That data currently comes from a database and it cannot fit entirely in memory. I would like to provide the user with an iterator that behaves like regular STL iterators, but that would be lazy. More precisely, I would be able to do something like that :

for (auto& item : big_bunch_of_data) {
    do_stuff_with(item);
}

With item being retrieved from the database only at each iteration. If I'm right, this new syntax is sugar for

for (stuff::iterator it = big_bunch_of_data.begin();it != big_bunch_of_data.end();it++) {
    do_stuff_with(*it);
}

Does it mean that by providing begin, end and operator++, I could have the desired behavior ? And, what are these methods supposed to do ? I mean, can I make them lazy without breaking stuff ?

Answer 1

Almost; the compiler will look in a few other places to get the begin and end iterators if it can't find begin or end methods on the container class; this is how range-based for loops work on arrays, that don't have begin and end members. It will also look for free functions begin and end by ADL, and eventually std::begin and std::end, so there's plenty of opportunity to retrofit range-based for loop support to existing containers. Section 6.5.4 covers the details.

For your other question, iterators absolutely can be lazy! A good example is std::istream_iterator which has to be lazy as it reads input from the console.

The requirement to use an iterator in a for loop is that it should satisfy the input iterator category, which is described in section 24.2.3; the required operations for that category are !=, unary *, and pre- and post-increment ++.

To let the language know that you've created an input iterator, you should inherit from std::iterator<std::input_iterator_tag, T, void, T *, T &> where T is the type your iterator deals in (section 24.4.3).