JMH - How to measure time it takes to insert 50M items in an ArrayList

I've an ArrayList of 50M, I would like to measure time it takes to store that many objects in it. It seems as all JMH modes are time based, we can't really control number of executions of code under @Benchmark. For examlpe, how can I ensure the following code is run exactly 50M times per fork?

@Benchmark
@BenchmarkMode(Mode.SampleTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@Fork(5)
public void run(BenchmarkState state) {
    try {
        state.queue.add(System.nanoTime());
    } catch (Exception e) {
        e.printStackTrace();
    }
}

You can create a benchmark class (ArrayListBenchmark) and a runner class (BenchmarkRunner).

• In ArrayListBenchmark class, you can add the benchmark method that iterates the desired number of times adding items to the List.
• In BenchmarkRunner class, you set the desired number of items to add to the List and config the runner options.

Note: Depending on your environment, adding 50M items may throw an OutOfMemoryError.

Benchmark class:

import java.util.List;
import java.util.ArrayList;

import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.Level;
import org.openjdk.jmh.annotations.Param;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.infra.Blackhole;

public class ArrayListBenchmark {

    @State(Scope.Thread)
    public static class ThreadState {

        @Param({})
        private int items;

        private List<Long> list;

        @Setup(Level.Iteration)
        public void setup() {
            list = new ArrayList<>();
        }
    }

    @Benchmark
    public void addItems(ThreadState state, Blackhole blackhole) {
        blackhole.consume(addItems(state.list, state.items));
    }

    private static boolean addItems(List<Long> list, int items) {
        for (int i = 0; i < items; i++) {
            list.add(System.nanoTime());
        }
        return true;
    }

}

Benchmark runner class:

import java.util.Locale;
import java.util.concurrent.TimeUnit;

import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.results.format.ResultFormatType;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import org.openjdk.jmh.runner.options.TimeValue;

public class BenchmarkRunner {

    private static final String ITEMS = "items";

    private static final String N_50_000_000 = "50000000";

    public static void main(String[] args) throws RunnerException {
        runArrayListBenchmark();
    }

    public static void runArrayListBenchmark() throws RunnerException {
        Options options = new OptionsBuilder()
                .include(ArrayListBenchmark.class.getSimpleName())
                .mode(Mode.AverageTime)
                .timeUnit(TimeUnit.NANOSECONDS)
                .warmupTime(TimeValue.seconds(1))
                .warmupBatchSize(1)
                .warmupIterations(5)
                .measurementTime(TimeValue.milliseconds(100))
                .measurementBatchSize(1)
                .measurementIterations(10)
                .param(ITEMS, N_50_000_000)
                .operationsPerInvocation(Integer.parseInt(N_50_000_000))
                .threads(1)
                .forks(5)
                .shouldFailOnError(true)
                .shouldDoGC(true)
                .resultFormat(ResultFormatType.CSV)
                .result("target/" + ArrayListBenchmark.class.getSimpleName().toLowerCase(Locale.ENGLISH) + ".csv")
                .build();
        new Runner(options).run();
    }

Output:

Result "ArrayListBenchmark.addItems":
  50.023 ±(99.9%) 0.768 ns/op [Average]
  (min, avg, max) = (48.094, 50.023, 53.020), stdev = 1.551
  CI (99.9%): [49.256, 50.791] (assumes normal distribution)

Benchmark                     (items)  Mode  Cnt   Score   Error  Units
ArrayListBenchmark.addItems  50000000  avgt   50  50.023 ± 0.768  ns/op