wrapping C callbacks with C++ lambdas, possible to use template polymorphism?

Question

Okay, I have posted a few questions lately related to wrapping a C callback API with a C++11-ish interface. I have almost got a satisfying solution, but I think it could be more elegant and need the help of some template metaprogramming wizards :)

Bear with me, as the example code is a little long, but I've tried to demonstrate the problem in one shot. Basically, the idea is that, given a list of function pointers and data context pointers, I want to provide a callback mechanism that can be provided with,

• Function pointers
• Function objects (functors)
• Lambdas

Moreover, I want to make these functions callable by a variety of prototypes. What I mean is, the C API provides about 7 different parameters to the callback, but in most cases the user code is really only interested in one or two of these. So I'd like the user to be able to specify only the arguments he is interested in. (This extends from the point of allowing lambdas in the first place... to allow conciseness.)

In this example, the nominal C callback takes an int and a float parameter, and an optional float* which can be used to return some extra data. So the intention of the C++ code is to be able to provide a callback of any of these prototypes, in any form that is "callable". (e.g. functor, lambda, etc.)

int callback2args(int a, float b);
int callback3args(int a, float b, float *c);

Here is my solution so far.

#include <cstdio>
#include <vector>
#include <functional>

typedef int call2args(int,float);
typedef int call3args(int,float,float*);
typedef std::function<call2args> fcall2args;
typedef std::function<call3args> fcall3args;

typedef int callback(int,float,float*,void*);
typedef std::pair<callback*,void*> cb;

std::vector<cb> callbacks;

template <typename H>
static
int call(int a, float b, float *c, void *user);

template <>
int call<call2args>(int a, float b, float *c, void *user)
{
    call2args *h = (call2args*)user;
    return (*h)(a, b);
}

template <>
int call<call3args>(int a, float b, float *c, void *user)
{
    call3args *h = (call3args*)user;
    return (*h)(a, b, c);
}

template <>
int call<fcall2args>(int a, float b, float *c, void *user)
{
    fcall2args *h = (fcall2args*)user;
    return (*h)(a, b);
}

template <>
int call<fcall3args>(int a, float b, float *c, void *user)
{
    fcall3args *h = (fcall3args*)user;
    return (*h)(a, b, c);
}

template<typename H>
void add_callback(const H &h)
{
    H *j = new H(h);
    callbacks.push_back(cb(call<H>, (void*)j));
}

template<>
void add_callback<call2args>(const call2args &h)
{
    callbacks.push_back(cb(call<call2args>, (void*)h));
}

template<>
void add_callback<call3args>(const call3args &h)
{
    callbacks.push_back(cb(call<call3args>, (void*)h));
}

template<>
void add_callback<fcall2args>(const fcall2args &h)
{
    fcall2args *j = new fcall2args(h);
    callbacks.push_back(cb(call<fcall2args>, (void*)j));
}

template<>
void add_callback<fcall3args>(const fcall3args &h)
{
    fcall3args *j = new fcall3args(h);
    callbacks.push_back(cb(call<fcall3args>, (void*)j));
}

// Regular C-style callback functions (context-free)
int test1(int a, float b)
{
    printf("test1 -- a: %d, b: %f", a, b);
    return a*b;
}

int test2(int a, float b, float *c)
{
    printf("test2 -- a: %d, b: %f", a, b);
    *c = a*b;
    return a*b;
}

void init()
{
    // A functor class
    class test3
    {
    public:
        test3(int j) : _j(j) {};
        int operator () (int a, float b)
        {
            printf("test3 -- a: %d, b: %f", a, b);
            return a*b*_j;
        }
    private:
        int _j;
    };

    // Regular function pointer of 2 parameters
    add_callback(test1);

    // Regular function pointer of 3 parameters
    add_callback(test2);

    // Some lambda context!
    int j = 5;

    // Wrap a 2-parameter functor in std::function
    add_callback(fcall2args(test3(j)));

    // Wrap a 2-parameter lambda in std::function
    add_callback(fcall2args([j](int a, float b)
                 {
                     printf("test4 -- a: %d, b: %f", a, b);
                     return a*b*j;
                 }));

    // Wrap a 3-parameter lambda in std::function
    add_callback(fcall3args([j](int a, float b, float *c)
                 {
                     printf("test5 -- a: %d, b: %f", a, b);
                     *c = a*b*j;
                     return a*b*j;
                 }));
}

int main()
{
    init();

    auto c = callbacks.begin();
    while (c!=callbacks.end()) {
        float d=0;
        int r = c->first(2,3,&d,c->second);
        printf("  result: %d (%f)\n", r, d);
        c ++;
    }
}

Okay, as you can see, this actually works. However, I find the solution of having to explicitly wrap the functors/lambdas as std::function types kind of inelegant. I really wanted to make the compiler match the function type automatically but this doesn't seem to work. If I remove the 3-parameter variant, then the fcall2args wrapper is not needed, however the presence of the fcall3args version of add_callback makes it apparently ambiguous to the compiler. In other words it seems to not be able to do pattern matching based on the lambda call signature.

A second problem is that I'm of course making copies of the functor/lambda objects using new, but not deleteing this memory. I'm not at the moment sure what the best way will be to track these allocations, although I guess in a real implementation I could track them in an object of which add_callback is a member, and free them in the destructor.

Thirdly, I don't find it very elegant to have specific types call2args, call3args, etc., for each variation of the callback I want to allow. It means I'll need an explosion of types for every combination of parameters the user might need. I was hoping there could be some template solution to make this more generic, but I am having trouble coming up with it.

Edit for explanation: The definition in this code, std::vector<std::pair<callback*,void*>> callbacks, is part of the problem definition, not part of the answer. The problem I am trying to solve is to map C++ objects onto this interface--therefore, proposing better ways to organize this std::vector doesn't solve the problem for me. Thanks. Just to clarify.

Edit #2: Okay, forget the fact that my example code uses std::vector<std::pair<callback*,void*>> callbacks to hold the callbacks. Imagine instead, as this is the actual scenario, that I have some C library implementing the following interface:

struct someobject *create_object();
free_object(struct someobject *obj);
add_object_callback(struct someobject *obj, callback *c, void *context);

where callback is,

typedef int callback(int a,float b,float *c, void *context);

Okay. So "someobject" will experience external events of some kind, network data, or input events, etc., and call its list of callbacks when these happen.

This is a pretty standard implementation of callbacks in C. Importantly, this is an existing library, something for which I cannot change, but I am trying to write a nice, idiomatic C++ wrapper around it. I want my C++ users to be able to add lambdas as callbacks. So, I want to design a C++ interface that allows users to be able to do the following:

add_object_callback(struct someobject *obj, func);

where func is one of the following:

1. a regular C function that doesn't use context.
2. a functor object
3. a lambda

Additionally, in each case, it should be possible for the function/functor/lambda to have either of the following signatures:

int cb2args(int a, float b);
int cb2args(int a, float b, float *c);

I think this should be possible, and I got about 80% of the way there, but I'm stuck on template polymorphism based on the call signature. I don't know offhand whether it's possible. Maybe it needs some voodoo involving function_traits or something, but it's a little beyond my experience. In any case, there are many, many C libraries that use such an interface, and I think it would be great to allow this kind of convenience when using them from C++.

Answer 1

Since you are using the C API in C++11, you could as well just wrap the whole thing in a C++ class. This is also necessary, as you mentioned in the 2nd problem, to solve the resource leak.

Also remember that a lambda expression without capture can be implicitly converted to a function pointer. This could remove all the call<*> because they can be moved into the add_callbacks.

And finally, we could use SFINAE to remove the fcall3args types. Here is the result.

class SomeObject {
    // The real object being wrapped.
    struct someobject* m_self;

    // The vector of callbacks which requires destruction. This vector is only a
    // memory store, and serves no purpose otherwise.
    typedef std::function<int(int, float, float*)> Callback;
    std::vector<std::unique_ptr<Callback>> m_functions;

    // Add a callback to the object. Note the capture-less lambda.
    template <typename H>
    void add_callback_impl(H&& h) {
        std::unique_ptr<Callback> callback (new Callback(std::forward<H>(h)));

        add_object_callback(m_self, [](int a, float b, float* c, void* raw_ctx) {
            return (*static_cast<Callback*>(raw_ctx))(a, b, c);
        }, callback.get());

        m_functions.push_back(std::move(callback));
    }

public:
    SomeObject() : m_self(create_object()) {}
    ~SomeObject() { free_object(m_self); }

    // We create 4 public overloads to add_callback:

    // This only accepts function objects having 2 arguments.
    template <typename H>
    auto add_callback(H&& h) -> decltype(h(1, 10.f), void()) {
        using namespace std::placeholders;
        add_callback_impl(std::bind(std::forward<H>(h), _1, _2));
    }

    // This only accepts function objects having 3 arguments.
    template <typename H>
    auto add_callback(H&& h) -> decltype(h(1, 1.0f, (float*)0), void()) {
        add_callback_impl(std::forward<H>(h));
    }

    // This only accepts function pointers. 
    void add_callback(int(*h)(int, float)) const {
        add_object_callback(m_self, [](int a, float b, float* c, void* d) {
            return reinterpret_cast<int(*)(int, float)>(d)(a, b);
        }, reinterpret_cast<void*>(h));
    }

    // This only accepts function pointers.
    void add_callback(int(*h)(int, float, float*)) const {
        add_object_callback(m_self, [](int a, float b, float* c, void* d) {
            return reinterpret_cast<int(*)(int, float, float*)>(d)(a, b, c);
        }, reinterpret_cast<void*>(h));
    }

    // Note that the last 2 overloads violates the C++ standard by assuming
    // sizeof(void*) == sizeof(func pointer). This is valid in POSIX, though.

    struct someobject* get_raw_object() const {
        return m_self;
    }
};

So the init() becomes:

void init(SomeObject& so) {
    // A functor class
    class test3 { ... };

    so.add_callback(test1);
    so.add_callback(test2);

    // Some lambda context!
    int j = 5;

    so.add_callback(test3(j));
    so.add_callback([j](int a, float b) -> int {
        printf("test4 -- a: %d, b: %f", a, b);
        return a*b*j;
    });

    so.add_callback([j](int a, float b, float *c) -> int {
        printf("test5 -- a: %d, b: %f", a, b);
        *c = a*b*j;
        return a*b*j;
    });
}

---

The full testing code (I'm not putting that to ideone here, because g++ 4.5 doesn't support implicitly converting a lambda to a function pointer, nor the range-based for.)

#include <vector>
#include <functional>
#include <cstdio>
#include <memory>

struct someobject;
struct someobject* create_object(void);
void free_object(struct someobject* obj);
void add_object_callback(struct someobject* obj,
                         int(*callback)(int, float, float*, void*),
                         void* context);

class SomeObject {
    // The real object being wrapped.
    struct someobject* m_self;

    // The vector of callbacks which requires destruction. This vector is only a
    // memory store, and serves no purpose otherwise.
    typedef std::function<int(int, float, float*)> Callback;
    std::vector<std::unique_ptr<Callback>> m_functions;

    // Add a callback to the object. Note the capture-less lambda.
    template <typename H>
    void add_callback_impl(H&& h) {
        std::unique_ptr<Callback> callback (new Callback(std::forward<H>(h)));

        add_object_callback(m_self, [](int a, float b, float* c, void* raw_ctx) {
            return (*static_cast<Callback*>(raw_ctx))(a, b, c);
        }, callback.get());

        m_functions.push_back(std::move(callback));
    }

public:
    SomeObject() : m_self(create_object()) {}
    ~SomeObject() { free_object(m_self); }

    // We create 4 public overloads to add_callback:

    // This only accepts function objects having 2 arguments.
    template <typename H>
    auto add_callback(H&& h) -> decltype(h(1, 10.f), void()) {
        using namespace std::placeholders;
        add_callback_impl(std::bind(std::forward<H>(h), _1, _2));
    }

    // This only accepts function objects having 3 arguments.
    template <typename H>
    auto add_callback(H&& h) -> decltype(h(1, 1.0f, (float*)0), void()) {
        add_callback_impl(std::forward<H>(h));
    }

    // This only accepts function pointers. 
    void add_callback(int(*h)(int, float)) const {
        add_object_callback(m_self, [](int a, float b, float* c, void* d) {
            return reinterpret_cast<int(*)(int, float)>(d)(a, b);
        }, reinterpret_cast<void*>(h));
    }

    // This only accepts function pointers.
    void add_callback(int(*h)(int, float, float*)) const {
        add_object_callback(m_self, [](int a, float b, float* c, void* d) {
            return reinterpret_cast<int(*)(int, float, float*)>(d)(a, b, c);
        }, reinterpret_cast<void*>(h));
    }

    // Note that the last 2 overloads violates the C++ standard by assuming
    // sizeof(void*) == sizeof(func pointer). This is required in POSIX, though.

    struct someobject* get_raw_object() const {
        return m_self;
    }
};

//------------------------------------------------------------------------------

int test1(int a, float b) {
    printf("test1 -- a: %d, b: %f", a, b);
    return a*b;
}

int test2(int a, float b, float *c) {
    printf("test2 -- a: %d, b: %f", a, b);
    *c = a*b;
    return a*b;
}

void init(SomeObject& so) {
    // A functor class
    class test3
    {
    public:
        test3(int j) : _j(j) {};
        int operator () (int a, float b)
        {
            printf("test3 -- a: %d, b: %f", a, b);
            return a*b*_j;
        }

    private:
        int _j;
    };

    so.add_callback(test1);
    so.add_callback(test2);

    // Some lambda context!
    int j = 5;

    so.add_callback(test3(j));
    so.add_callback([j](int a, float b) -> int {
        printf("test4 -- a: %d, b: %f", a, b);
        return a*b*j;
    });

    so.add_callback([j](int a, float b, float *c) -> int {
        printf("test5 -- a: %d, b: %f", a, b);
        *c = a*b*j;
        return a*b*j;
    });
}

//------------------------------------------------------------------------------

struct someobject {
    std::vector<std::pair<int(*)(int,float,float*,void*),void*>> m_callbacks;
    void call() const {
        for (auto&& cb : m_callbacks) {
            float d=0;
            int r = cb.first(2, 3, &d, cb.second);
            printf("  result: %d (%f)\n", r, d);
        }
    }
};

struct someobject* create_object(void) {
    return new someobject;
}

void free_object(struct someobject* obj) {
    delete obj;
}

void add_object_callback(struct someobject* obj,
                         int(*callback)(int, float, float*, void*),
                         void* context) {
    obj->m_callbacks.emplace_back(callback, context);
}

//------------------------------------------------------------------------------

int main() {
    SomeObject so;
    init(so);
    so.get_raw_object()->call();
}