generating huge Boost.MPL type sequences

Question

Consider the following code that automates generation of Boost.MPL type sequences (list or vector).

    #include <iostream>                     // cout
    #include <boost/mpl/for_each.hpp>       // for_each
    #include <boost/mpl/identity.hpp>       // identity, make_identity
    #include <boost/mpl/int.hpp>            // int_
    #include <boost/mpl/list.hpp>           // list
    #include <boost/mpl/next.hpp>           // next
    #include <boost/mpl/push_front.hpp>     // push_front
    #include <boost/mpl/vector.hpp>         // vector

    template<size_t, typename> struct iota_n;

    template<typename Value>
    struct iota_n<0, Value>
    :
            boost::mpl::list<>      // can change this to boost::mpl::vector<>
    {};

    template<size_t N, typename Value>
    struct iota_n
    :
            boost::mpl::push_front< typename
                    iota_n< 
                            N - 1, typename
                            boost::mpl::next<Value>::type
                    >::type,
                    Value
            >
    {};

    // works for N <=  20 and boost::mpl::vector
    // works for N <= 247 and boost::mpl::list
    typedef iota_n< 247, boost::mpl::int_<0> >::type sequence;

    struct print
    {
            template<typename T>
            void operator()(boost::mpl::identity<T>)
            {
                    std::cout << T::value << "\n";
            }
    };

    int main()
    {
            boost::mpl::for_each<sequence, boost::mpl::make_identity<> >(
                    print()
            );
            std::cout << BOOST_MPL_LIMIT_LIST_SIZE << '\n';         // 20 on my system
            std::cout << BOOST_MPL_LIMIT_VECTOR_SIZE << '\n';       // 20 on my system
            return 0;
    }

According to the Boost.MPL documentation, a boost::mpl::list sequence can have up to BOOST_MPL_LIMIT_LIST_SIZE elements, and similarly for a boost::mpl::vector the compiler can go up to BOOST_MPL_LIMIT_VECTOR_SIZE. Both macros evaluate to 20 on my system.

MSVC++ 2010 and Boost 1.47.0 indeed cannot generate vectors with more than the documented 20 elements. What is surprising that it can, however, generate lists with up to 247 elements!

Does anyone know why this happens?

Answer 1

As per the docs, BOOST_MPL_LIMIT_xxx_SIZE specifies the limit for the sequence's variadic form (e.g. list<>); the numbered form (e.g. list42<>), has no predefined top limit, aside from compiler limitations on the number of template parameters. Okay, the latter statement is not completely accurate: in practice, in the default library configuration, there is a limit on the numbered forms imposed by using pre-generated preprocessed headers; see this post on how to lift it.

FOLLOW UP: @rhalbersma You seem to be bundling together two separate notions: the maximum number of list elements vs. the maximum arity of the list's "constructor". BOOST_MPL_LIMIT_LIST_SIZE controls the latter, not the former, and there is really no dependency between the two. Your code above is testing the former; maximum template arity is a totally different beast.

The reason there is an arity limit for MPL sequences in the first place is that the library had to emulate variadic templates (it was written way before C++11), which is typically done by defaulting unused arguments to some auxiliary type and providing a bunch of specializations to weed out those unused arguments before constructing the actual sequence. The cost of this is that the default arguments typically show up in error messages and obscure everything else, and a large number of specializations has noticeable effect on compilation times. IOW, you had to stop somewhere, and at the time it seemed unlikely that you'd often need to pass more than 20 sequence elements to a sequence's "constructor" (and if you did, there are always numbered forms), hence the current limits.