How to benchmark custom interface using google benchmark

since I can't find really good resource or documentation besides what's on the github readme, I'll ask the community here.

Assuming that I have an interface called Base, and multiple derived types.
I want to benchmark a virtual function Foo of all derived types with the same function and the same argument passed to Foo.
For each derived type I want to call Foo multiple times in a single benchmark with varying arguments passed from an std::vector<std::size_t>.

Also some of my constructor of derived types takes extra arguments, how should I deal with it?

The easiest thing to do as I see it, is to use templated benchmark and pass to:

const std::vector<std::size_t> v {1, 2, 4, 8, 16};
BENCHMARK_TEMPLATE(func, Derived1, v);
BENCHMARK_TEMPLATE(func, Derived2, v);
BENCHMARK_TEMPLATE(func, Derived3, v);
...

I don't feel this is the correct way to do this since for each derived type with different constructor signature I would need to have different benchmark function which is almost identical to the previous one.

What would be the proper way to benchmark this?

**UPDATE*: The way I solved this is as follows:

std::unique_ptr<Base> GetObj(const std::size_t index)
{
    switch(index)
    {
    case 0:
        return std::make_unique<Derived1>(...);
    case 1:
        return std::make_unique<Derived2>(...);
    ...
    }
}

void func(benchmark::State& state)
{
    auto r = state.range();
    auto object = GetObj(r);
    for (auto _ : state)
    {
        ...
    }
}
BENCHMARK(func)->DenseRange(0,5);

But what I don't like about it is that the name of the benchmark tests is of the pattern func/0-5 and I would like it to be func/Derived1, func/Derived2 and so on.. any ideas how to achieve that?

Solution

Your approach is pretty much correct. If you want to avoid the multiple benchmark functions, move the core of the benchmark to a separate function and call it from each of the per-type ones. Something like:

static void Benchmark(Base* b, benchmark::State& st) {
  for (auto _ : state) {
    b->DoTheThing()
  }
}

void BM_Foo(benchmark::State& st) {
  Foo f;
  Benchmark(&f, st);
}

BENCHMARK(BM_Foo);

If you want to be more sophisticated, you could use the RegisterBenchmark function to register things more flexibly. Something like:

auto BM_base = [](benchmark::State& st, auto inputs, Base* b) { /* ... */ };

int main(int argc, char**argv) {
  const std::vector<std::size_t> v = {1, 2, 4, 8, 16};
  benchmark::RegisterBenchmark("foo", BM_base, v, new Foo());
  benchmark::Initialize(&argc, argv);
  return benchmark::RunSpecifiedBenchmarks();
}

As an aside, should v be set as the Args when the benchmark is registered?