c++parametersbenchmarkingmicrobenchmarkgoogle-benchmark

How to change the increment or step or scale of a Google Benchmark 'Range()' function


I have a very simple Google Benchmark program that benchmarks a function taking two integer arguments, I'm trying to use the benchmark to see how exactly does the time the function takes increase as the second argument's value increases from 1 to 100, so with the first argument staying with the same value of 999999 .

The way to achieve this that looked the most logical to me was to use Google Benchmark's Ranges() function like this:

// registering function 'largestDivisorOdd' as a benchmark
BENCHMARK(largestDivisorOdd) ->Ranges( { {999999, 999999}, {1,100} } );

The resulting output was this:

-----------------------------------------------------------------------
Benchmark                             Time             CPU   Iterations
-----------------------------------------------------------------------
largestDivisorOdd/999999/1          116 ns          116 ns      6018141
largestDivisorOdd/999999/8          205 ns          205 ns      3485489
largestDivisorOdd/999999/64        2715 ns         2715 ns       260611
largestDivisorOdd/999999/100       2710 ns         2710 ns       256160

My problem is that it seems Google Benchmark has made the range from only values of an exponential increase from 1 to 100, resulting in only 4 benchmarks of 4 different values for the second parameter, instead of 100 benchmarks of 100 values from 1 to 100 as I expected and wanted.


Solution

  • One of the following should work:

    BENCHMARK(largestDivisorOdd)
        ->ArgsProduct({
          benchmark::CreateRange(999999, 999999, /*multi=*/2), // This is probably not what you want
          benchmark::CreateDenseRange(1, 100, /*step=*/1) // This creates a DenseRange from 1 to 100
        })
    

    Or create your own custom arguments:

    static void CustomArguments(benchmark::internal::Benchmark* b) {
      for (int i = 999999; i <= 999999; ++i)
        for (int j = 1; j <= 100; j++)
          b->Args({i, j});
    }
    BENCHMARK(BM_SetInsert)->Apply(CustomArguments);
    

    Both examples are taken from the User Guide