Detecting the parameter types in a Spirit semantic action

Question

General case: I can't figure out why my Spirit grammar/semantics actions aren't compiling.

Sometimes, the compiler will complain about assignment or type incompatibilities and I have no clue what's wrong. The problem occurs in two main areas:

• predicting the type of synthesized attributes for a rule/expression
  • consequently, predicting what types of attributes can be legally defined as the exposed attribute for the rule (relying in builting conversions, fusion adaptors or Spirit customization points)
• matching the argument types for my semantic action so that
  • the compiler will be able to compile the function invocation
  • the invocation will not invoke unnecessary implicit conversions in the process

The compiler error is not exactly tractable, and either the documentation is wrong, or I misunderstood it.

Is there a way to find out exactly what Spirit passes into my semantic action, anyway?

Example code:

struct mybase             { int a,b; };
struct myderived : mybase { int c,d; };

BOOST_FUSION_ADAPT_STRUCT(mybase,    (int,a)(int,b));
BOOST_FUSION_ADAPT_STRUCT(myderived, (int,a)(int,b)(int,c)(int,d));

auto base_expr = int_ >> int_; // avoids assigning to struct attribute

rule<decltype(f), mybase()   , space_type> base_       = int_ >> int_;
rule<decltype(f), myderived(), space_type> derived_    = base_ >> int_ >> int_;

myderived data;
bool ok = phrase_parse(f,l,derived_,space,data);

This code won't compile, with a huge amount of impenetrable errors.

(loosely adapted from a posting on the spirit-general list)

Answer 1

I could solve the problem for this particular case (in fact we discussed options on the list), but really, this kind of 'enigmatic' error creeps up more often with Boost Spirit and it would be nice to get a handle on the general class of problems.

Your first resource should be the excellent spirit documentation, which details exactly what the synthesized attribute will be for a given parser primitive, operator or directive. See the Reference section to Spirit Qi Docs.

In some cases, I have taken to shifting the focus from 'trying to pry the information from the compiler error list' to 'actively querying Spirit for the types it passes'. The technique I use for this is the Polymorphic Callable Type (see Spirit/Fusion docs).

Here is one that uses GCC specific APIs to pretty [sic] print the types it detects:

Functor what_is_the_attr

#include <cxxabi.h>
#include <stdlib.h>
#include <string>
#include <iostream>

template <typename T> std::string nameofType(const T& v) {
    int     status;
    char   *realname = abi::__cxa_demangle(typeid(v).name(), 0, 0, &status);
    std::string name(realname? realname : "????");
    free(realname);
    return name;
}

struct what_is_the_attr {
    template <typename> struct result { typedef bool type; };

    template <typename T> bool operator()(T& attr) const {
        std::cerr << "what_is_the_attr: " << nameofType(attr) << std::endl;
        return true;
    }
};

Sample use: detect synthesized attribute type

You can use it to detect exactly what type the synthesized attribute type of a parser expression actually ends up being:

template <typename Exp>
    void detect_attr_type(const Exp& exp)
{
    using namespace boost::spirit::qi;

    const char input[] = "1 2 3 4";
    auto f(std::begin(input)), l(std::end(input)-1);

    bool dummy = phrase_parse(
            f, l, 
            exp [ what_is_the_attr() ],
            space);
}

(Note: this shows a limitation of the approach - the technique assumes you have an 'otherwise' working grammar and you know how to pass an input that satisfies the expression enough to trigger the semantic action. In most cases, this will be true when you are hacking on your Spirit parser, though)

Let's test it. E.g. let's see what the difference is between and expression of medium complexity, and the same wrapped inside a qi::raw[] directive:

int main()
{
    detect_attr_type(       -(int_ >> *int_)    );
    detect_attr_type( raw [ -(int_ >> *int_) ] );
}

Output:

what_is_the_attr: boost::optional<boost::fusion::vector2<int, std::vector<int, std::allocator<int> > > >
what_is_the_attr: boost::iterator_range<char const*>

At the bottom, we will apply this to the question in the OP.

Sample use: detecting types passed into Semantic Actions

We could use the same unary function object (what_is_the_attr) to detect these, however, semantic actions can take any number of arguments, so we need to generalize. This would be tedious work, if it weren't for variadic template (woot! for c++0x):

struct what_are_the_arguments {
    template <typename...> struct result { typedef bool type; };

    template <typename... T> bool operator()(const T&... attr) const {
        std::vector<std::string> names { nameofType(attr)... };
        std::cerr << "what_are_the_arguments:\n\t";
        std::copy(names.begin(), names.end(), std::ostream_iterator<std::string>(std::cerr, "\n\t"));
        std::cerr << '\n';
        return true;
    }
};

Repeated for the above test cases reveals that Spirit actually tries to call the semantic action with three arguments if possible (as documented):

what_are_the_arguments:
    boost::optional<boost::fusion::vector2<int, std::vector<int, std::allocator<int> > > >
    boost::spirit::unused_type
    bool

what_are_the_arguments:
    boost::iterator_range<char const*>
    boost::spirit::unused_type
    bool

But the nice thing is, that you can now apply this to any semantic action:

template <typename ExpWSA> void test(const ExpWSA& exp)
{
    const char input[] = "1 2 3 4";
    auto f(std::begin(input)), l(std::end(input)-1);

    qi::phrase_parse(f, l, exp, qi::space);
}

int main()
{
    test(-(-double_ >> *int_) [ phx::bind(what_are_the_arguments(), _1, _2, _0, phx::ref(std::cout), 42) ]);
}

Printing, for this (sorry) very contrived example:

what_are_the_arguments:
    boost::optional<double>
    std::vector<int, std::allocator<int> >
    boost::fusion::vector2<boost::optional<double>, std::vector<int, std::allocator<int> > >
    std::ostream
    int

Applied to the OP

The synthesized attribute of the derived rule is not the same as for int_>>int_>>int_>>int_:

auto base_expr = int_ >> int_; // avoids assigning to struct attribute

rule<const char*, mybase(), space_type> base_       = base_expr;

test(base_     >> int_ >> int_ [ what_is_the_attr() ] );
test(base_expr >> int_ >> int_ [ what_is_the_attr() ] );

Will print

what_is_the_attr: boost::fusion::vector3<mybase, int, int>
what_is_the_attr: boost::fusion::vector4<int, int, int, int>

There is your problem. We discussed some workarounds based on this diagnostic in the original thread (and see the other answers here). But this post should help answering the general case question.

Full code listing

In integrated form, compiled with gcc 4.6.1 --std=c++0x and boost 1_48:

#include <cxxabi.h>
#include <iostream>
#include <iterator>
#include <stdlib.h>
#include <string>
#include <vector>

template <typename T> std::string nameofType(const T& v)
{
    int     status;
    char   *realname = abi::__cxa_demangle(typeid(v).name(), 0, 0, &status);
    std::string name(realname? realname : "????");
    free(realname);

    return name;
}

struct what_is_the_attr {
    template <typename> struct result { typedef bool type; };

    template <typename T> bool operator()(T& attr) const {
        std::cerr << "what_is_the_attr: " << nameofType(attr) << std::endl;
        return true;
    }
};

struct what_are_the_arguments {
    template <typename...> struct result { typedef bool type; };

    template <typename... T> bool operator()(const T&... attr) const {
        std::vector<std::string> names { nameofType(attr)... };
        std::cerr << "what_are_the_arguments:\n\t";
        std::copy(names.begin(), names.end(), std::ostream_iterator<std::string>(std::cerr, "\n\t"));
        std::cerr << '\n';
        return true;
    }
};

#include <boost/fusion/adapted.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <boost/spirit/include/qi.hpp>

struct mybase             { int a,b; };
struct myderived : mybase { int c,d; };

BOOST_FUSION_ADAPT_STRUCT(mybase,    (int,a)(int,b));
BOOST_FUSION_ADAPT_STRUCT(myderived, (int,a)(int,b)(int,c)(int,d));

template <typename ExpWSA>
void test(const ExpWSA& exp)
{
    using namespace boost::spirit::qi;

    const char input[] = "1 2 3 4";
    auto f(std::begin(input)), l(std::end(input)-1);

    bool dummy = phrase_parse(f, l, exp, space);
}

int main()
{
    using namespace boost::spirit::qi;

    // Diagnostics for the OP case
    auto base_expr = int_ >> int_;                                   // avoids assigning to struct attribute
    rule<const char*, mybase(), space_type> base_       = base_expr;

    // Derived rule, different formulations
    test((base_     >> int_ >> int_) [ what_is_the_attr() ] );
    test((base_expr >> int_ >> int_) [ what_is_the_attr() ] );

    // Applied to attribute types
    test(raw [ -(int_ >> *int_) ]  [ what_is_the_attr() ] );
    test(-(int_ >> *int_)          [ what_is_the_attr() ] );

    // applied to semantic actions - contrived example
    namespace phx = boost::phoenix;
    test(-(-double_ >> *int_) [ phx::bind(what_are_the_arguments(), _1, _2, _0, phx::ref(std::cout), 42) ]);

    return 0;
}