Parsing a C++ string into a tuple

Question

I am working on a simple CSV parser that would store the lines of a file in a tuple. This would be an easy task if it wasn't for the fact that the number of entries on the lines inside the file is a variable, as well as their type. Thus, the lines could be like that:

1,2.2,hello,18,world

The parser should be able to work like this:

ifstream file("input.csv");
SimpleCSVParser<int, float, string, int, string> parser(file);

Things get complex when I try to implement a function to parse the actual line. I still have not found a way to extract the next type from the parameter list to declare the variable before calling file >> var on it. I would also need to do this in a loop, somehow constructing a tuple from the results of each iteration.

So how do I parse the string into a tuple using plain C++11? I tried this:

template <typename ...Targs>
tuple<Targs...> SimpleCSVParser<Targs...>::iterator::operator*() {
    istringstream in(cur);
    in.imbue(locale(locale(), new commasep)); // for comma separation
    tuple<Targs...> t;
    for (size_t i = 0; i < sizeof...(Targs); ++i) {
        tuple_element<i,decltype(t)>::type first;
        in >> first;
        auto newt = make_tuple(first);
        // what do I do here?
    }
}

But it doesn't work since the tuple I use to extract types is empty.

Answer 1

It seems, you try to iterate over tuple indices/types which doesn't work, I think. What you can do, however, is to just call a read function for each member. The idea is to delegate processing of the tuple to a function which uses a parameter pack to expand an operation to an operation on each element. std::index_sequence<...> can be used to get the sequence of integers.

Something like this:

template <typename T>
bool read_tuple_element(std::istream& in, T& value) {
    in >> value;
    return true;
}

template <typename Tuple, std::size_t... I>
void read_tuple_elements(std::istream& in, Tuple& value, std::index_sequence<I...>) {
    std::initializer_list<bool>{ read_tuple_element(in, std::get<I>(value))... });
}

template <typename ...Targs>
tuple<Targs...> SimpleCSVParser<Targs...>::iterator::operator*() {
    std::istringstream in(cur);
    in.imbue(std::locale(std::locale(), new commasep)); // for comma separation
    std::tuple<Targs...> t;
    read_tuple_elements(in, t, std::make_index_sequence<sizeof...(Targs)>{});
    if (in) { // you may want to check if all data was consumed by adding && in.eof()
        // now do something with the filled t;
    }
    else {
        // the value could *not* successfully be read: somehow deal with that
    }
}

The basic idea of the above code is simply to create a suitable sequence of calls to read_tuple_element(). Before jumping into the generic code, assume we'd want to implement reading of a std::tuple<T0, T1, T2> value with just three elements. We could implement the read using (using rte() instead of read_tuple_element() for brevity):

rte(get<0>(value)), rte(get<1>(value)), rte(get<2>(value));

Now, instead of writing this out for each number of elements, if we had an index sequence std::size_t... I we could get this sequence [nearly] using

rte(get<I>(value))...;

It isn't allowed to expand a parameter pack like this, though. Instead, the parameter pack needs to be put into some context. The code above uses a std::initializer_list<bool> for this purpose: the elements of a std::initializer_list<T> are constructed in the order listed. That is, we got

std::initializer_list<bool>{ rte(get<I>(value))... };

The missing bit is how to create the parameter pack I evaluating to a sequence of suitable indices. Conveniently, the standard library defines std::make_index_sequence<Size> which creates a std::index_sequence<I...> with a sequence of values for I as 0, 1, 2, ..., Size-1. So, calling read_tuple_elements() with std::make_index_sequence<sizeof...(Targs){} creates an object with a suitable list of arguments which can be deduced and then used to expand the tuple into a sequence of elements passed to read_tuple_element().