C++ Iterator Pipelining Designs

Question

Suppose we want to apply a series of transformations, int f1(int), int f2(int), int f3(int), to a list of objects. A naive way would be

SourceContainer source;

TempContainer1 temp1;
transform(source.begin(), source.end(), back_inserter(temp1), f1);
TempContainer2 temp2;
transform(temp1.begin(), temp1.end(), back_inserter(temp2), f2);

TargetContainer target;
transform(temp2.begin(), temp2.end(), back_inserter(target), f3);

This first solution is not optimal because of the extra space requirement with temp1 and temp2. So, let's get smarter with this:

int f123(int n) { return f3(f2(f1(n))); }
...
SourceContainer source;
TargetContainer target;
transform(source.begin(), source.end(), back_inserter(target), f123);

This second solution is much better because not only the code is simpler but more importantly there is less space requirement without the intermediate calculations.

However, the composition f123 must be determined at compile time and thus is fixed at run time.

How would I try to do this efficiently if the composition is to be determined at run time? For example, if this code was in a RPC service and the actual composition--which can be any permutation of any subset of f1, f2, and f3--is based on arguments from the RPC call.

Answer 1

EDIT: Working version at http://ideone.com/5GxnW . The version below has the ideas but does not compile. It supports run time type checking, and run time function composition.

The idea is to define a generic (unary) function class, and a way to compose them with run time type checks. This is done with a combination of boost::any, boost::function and the type erasure idiom.

#include <boost/any.hpp>
#include <boost/function.hpp>
#include <boost/shared_ptr.hpp>


template <typename T>
struct identity
{
    T operator()(const T& x) { return x; }
};

struct any_function
{
    template <typename Res, typename Arg>
    any_function(boost::function<Res, Arg> f)
    {
        impl = make_impl(f);
    }

    boost::any operator()(const boost::any& x)
    {
        return impl->invoke(x);
    }

    static any_function compose(const any_function& f,
                                const any_function& g)
    {
        any_function ans;
        ans.impl = compose_impl(f.impl, g.impl);
        return ans;
    }

    template <typename T>
    static any_function id()
    {
        using boost::function
        return any_function(function<T(T)>(identity<T>()));
    }

    template <typename Res, typename Arg>
    boost::function<Res(Arg)> to_function()
    {
        using boost::function;
        return function<Res(Arg)>(to_function_helper(impl));
    }

private:
    any_function() {}

    struct impl_type
    {
        virtual ~impl_type() {}
        virtual boost::any invoke(const boost::any&) = 0;
    };

    boost::shared_ptr<impl_type> impl;

    template <typename Res, typename Arg>
    static impl_type* make_impl(boost::function<Res(Arg)> f)
    {
        using boost::function;
        using boost::any;
        using boost::any_cast;

        class impl : public impl_type
        {
            function<Res(Arg)> f;

            any invoke(const any& x)
            {
                const Arg& a = any_cast<Arg>(x);
                return any(f(a));
            }

        public:
            impl(function<Res(Arg)> f) : f(f) {}
        };

        return new impl(f);
    }

    impl_type* compose_impl(boost::shared_ptr<impl_type> f,
                            boost::shared_ptr<impl_type> g)
    {
        using boost::any;
        using boost::shared_ptr;

        class impl : public impl_type
        {
            shared_ptr<impl> f, g;

            any invoke(const any& x)
            {
                return g->invoke(f->invoke(x));
            }

        public:
            impl(const shared_ptr<impl>& f,
                 const shared_ptr<impl>& g)
                : f(f), g(g)
            {}
        };

        return new impl(f, g);
    }

    struct to_function_helper
    {
        template <typename Res, typename Arg>
        Res operator()(const Arg& x)
        {
            using boost::any;
            using boost::any_cast;

            return any_cast<Res>(p->invoke(any(x)));
        }

        to_function_helper(const boost::shared_ptr<impl>& p) : p(p) {}

    private:
        boost::shared_ptr<impl> p;
    };
};

Now, let's use standard algorithms and do this (this even works on empty sequences):

// First function passed is evaluated first. Feel free to change.
template <typename Arg, typename Res, typename I>
boost::function<Res(Arg)> pipeline(I begin, I end)
{
    return std::accumulate(begin, end, 
        any_function::id<Arg>,
        std::ptr_fun(any_function::compose)
    ).to_function<Res, Arg>();
}

and use the following to apply it

std::vector<any_function> f;
std::vector<double> v;
std::vector<int> result;

std::transform(v.begin(), v.end(), 
    result.begin(), 
    pipeline<double, int>(f.begin(), f.end())
);

You can even use boost::transform_iterator

typedef boost::transform_iterator<
    boost::function<double, int>, 
    std::vector<double>::const_iterator
> iterator;

boost::function<double, int> f = pipeline<double, int>(f.begin(), f.end());
std::copy(iterator(v.begin(), f), iterator(v.end(), f), result.begin());

Answer 2

template<class T>
class compose {
    typedef T (*f)(T);

    f first_func;
    f second_func;

public:

    compose(f one,f two) :
        first_func(one),
        second_func(two)        
    {}

    T operator()(T const &input) {
        T temp = first_func(input);
        return second_func(temp);
    }
};

#ifdef TEST

int f(int x) { return 8 + x; }
int g(int x) { return 2 * x; }
int h(int x) { return x * x; }

#include <iostream>

int main(int argc, char **argv) {
    compose<int> x(f, g);
    compose<int> y(g, f);

    std::cout << x(6) << std::endl;
    std::cout << y(6) << std::endl;

    typedef int (*func)(int);

    func funcs[] = {f, g, h};

    compose<int> z(funcs[atoi(argv[1])], funcs[atoi(argv[2])]);
    std::cout << z(6);

    return 0;
}

#endif

With C++0x, we should be able to use auto to eliminate having to specify the argument/return type. For the moment I've assumed they're the same, though in theory, you might like the ability to include conversions in the mix.