google/benchmark: BENCHMARK_TEMPLATE syntax

Question

I want to run a google/benchmark on float, double and long double.

Given the BENCHMARK_TEMPLATE example, I attempted the following:

#include <cmath>
#include <ostream>
#include <random>
#include <benchmark/benchmark.h>

template<typename Real>
BM_PowTemplated(benchmark::State& state) {
    std::random_device rd;
    std::mt19937 gen(rd());
    std::uniform_real_distribution<Real> dis(1, 10);
    auto s = dis(gen);
    auto t = dis(gen);
    Real y;
    while (state.KeepRunning()) {
        benchmark::DoNotOptimize(y = std::pow(s, t));
    }
    std::ostream cnull(0);
    cnull << y;
}

BENCHMARK_TEMPLATE(BM_PowTemplated, float, double, long double);
BENCHMARK_MAIN();

I imagined that this would then create three benchmarks for float, double and long double, but instead it doesn't compile!

What is the proper syntax for creating templated google benchmarks? Is my mental model for how BENCHMARK_TEMPLATE should work correct and if so how can I fix this code?

Answer 1

You use BENCHMARK_TEMPLATE in wrong way with your

BENCHMARK_TEMPLATE(BM_PowTemplated, float, double, long double);

The readme file of https://github.com/google/benchmark/ says

template <class Q> int BM_Sequential(benchmark::State& state) { .. }
BENCHMARK_TEMPLATE(BM_Sequential, WaitQueue<int>)->Range(1<<0, 1<<10);

Three macros are provided for adding benchmark templates.

#define BENCHMARK_TEMPLATE(func, ...) // Takes any number of parameters.

#define BENCHMARK_TEMPLATE1(func, arg1)
#define BENCHMARK_TEMPLATE2(func, arg1, arg2)

So, BENCHMARK_TEMPLATE with arg1 is used for functions with one template parameter, with arg1 and arg2 for functions with two template parameters. Your BM_PowTemplated has only one parameter, so you can't use BENCHMARK_TEMPLATE with 3 args.

Check test/cxx03_test.cc of google/benchmark for example: https://github.com/google/benchmark/blob/b2e734087532897b7bb4c51a6b4f503060c9a20f/test/cxx03_test.cc

template <class T, class U>
void BM_template2(benchmark::State& state) {
    BM_empty(state);
}
BENCHMARK_TEMPLATE2(BM_template2, int, long);

template <class T>
void BM_template1(benchmark::State& state) {
    BM_empty(state);
}
BENCHMARK_TEMPLATE(BM_template1, long);
BENCHMARK_TEMPLATE1(BM_template1, int);

PS: definition of macro: https://github.com/google/benchmark/blob/2d088a9f2d41acb77afc99d045f669e1a21b61ef/include/benchmark/benchmark_api.h#L684

We can see that a (a, b) arguments of macro goes inside < and >, and they are used as f< a,b >:

// template<int arg>
// void BM_Foo(int iters);
//
// BENCHMARK_TEMPLATE(BM_Foo, 1);
//
// will register BM_Foo<1> as a benchmark.
#define BENCHMARK_TEMPLATE1(n, a) \
  BENCHMARK_PRIVATE_DECLARE(n) =  \
      (::benchmark::internal::RegisterBenchmarkInternal( \
        new ::benchmark::internal::FunctionBenchmark(#n "<" #a ">", n<a>)))

#define BENCHMARK_TEMPLATE2(n, a, b)                     \
  BENCHMARK_PRIVATE_DECLARE(n) =                         \
      (::benchmark::internal::RegisterBenchmarkInternal( \
        new ::benchmark::internal::FunctionBenchmark(    \
            #n "<" #a "," #b ">", n<a, b>)))

#define BENCHMARK_TEMPLATE(n, ...)           \
  BENCHMARK_PRIVATE_DECLARE(n) =             \
      (::benchmark::internal::RegisterBenchmarkInternal( \
        new ::benchmark::internal::FunctionBenchmark( \
        #n "<" #__VA_ARGS__ ">", n<__VA_ARGS__>)))

So, BENCHMARK_TEMPLATE can't iterate over several variants and define several variants of you function from one line.