How to efficiently roll out a sequence using c++ templates

Question

I've a complex type C depending on a template parameter which I need in a (length bounded) sequence. A constexpr function next() is available to go from C_n -> C_n+1. As every sequence element has a different type I'm using a std::tuple to store the results. The mkTuple() functions take care of the (limited) sequence rollout.

Here's a simplified example of what I did (using std::array as placeholder for my more complex C):

#include <array>
#include <tuple>
#include <iostream>


template<std::size_t OFFSET>
using C=std::array<uint8_t, OFFSET>;

static
constexpr
std::size_t
next(const std::size_t START, const std::size_t DISTANCE)
{
    return START + DISTANCE;
}


template<template<std::size_t OFFSET> class CONTENT, std::size_t START, std::size_t DISTANCE, std::size_t SEQ_LENGTH>
struct mkTuple
{
    using _LT = CONTENT<START>;
    using _RT = typename mkTuple<CONTENT, next(START, DISTANCE), DISTANCE, SEQ_LENGTH - 1>::tuple;
    using tuple=decltype(std::tuple_cat(std::make_tuple(_LT()), _RT()));
};


template<template<std::size_t OFFSET> class CONTENT, std::size_t START, std::size_t DISTANCE>
struct mkTuple<CONTENT, START, DISTANCE, 1>
{
    using _LT = CONTENT<START>;
    using tuple = std::tuple<_LT>;
};


int
main()
{
    using tMyTuple = typename mkTuple<C, 16, 2, 64>::tuple;

    const std::size_t iTupleLength = std::tuple_size<tMyTuple>::value;
    std::cout << "Sequence length =  " << iTupleLength << std::endl;

    const tMyTuple myTuple;
    std::cout << "Tuple[4].size() =  " << std::get<4>(myTuple).size() << std::endl;

    return 0;
}

Problem is that my mkTuple() functions seem to be soo memory expensive that I quickly run into compiler-out-of-memory trouble. In fact g++ -ftime-report ... reports >1GB usage for template instantiations. And this number quickly (exponentially?) rises with growing sequence lengths.

I suspect the std::tuple instantiation required for std::tuple_cat to be the inflater. Does anybody have a good idea how to roll out a sequence more efficiently?

Thx in advance!

Answer 1

Seems like std::tuple is overkill for your scenario - you can work only on the type-system level and then convert to std::tuple at the end.

Since std::tuple's implementation is big and complicated, it is not surprising that it will be slow for a large number of instantiations. Roll your own typelist:

template <typename... Ts>
struct typelist 
{ 
    using as_tuple = std::tuple<Ts...>; 
};

template <typename... As, typename... Bs>
constexpr auto typelist_cat(typelist<As...>, typelist<Bs...>) 
    -> typelist<As..., Bs...> 
{
    return {};
}

Then replace usage of std::tuple:

template<template<std::size_t OFFSET> class CONTENT, std::size_t START, std::size_t DISTANCE, std::size_t SEQ_LENGTH>
struct mkTuple
{
    using _LT = CONTENT<START>;
    using _RT = typename mkTuple<CONTENT, START + DISTANCE, DISTANCE, SEQ_LENGTH - 1>::tuple;
    using tuple = decltype(typelist_cat(typelist<_LT>{}, _RT{}));
};


template<template<std::size_t OFFSET> class CONTENT, std::size_t START, std::size_t DISTANCE>
struct mkTuple<CONTENT, START, DISTANCE, 1>
{
    using _LT = CONTENT<START>;
    using tuple = typelist<_LT>;
};

Finally, convert at the end:

using tMyTuple = typename mkTuple<C, 16, 2, 64>::tuple::as_tuple;

live wandbox example

Output:

Sequence length = 64

Tuple[4].size() = 24