javajmh

How to benchmark assert statements?


The next version of my Design by Contract API uses the following syntax:

assert requireThat(name, value).size().isGreaterThan(3).elseThrowAssert();

The idea being that the validation only runs if assertions are enabled. I want to time four scenarios:

  1. Time an empty method.
  2. Time a validation without using assert.
  3. Time a validation using assert, with asserts are enabled.
  4. Time a validation using assert, with asserts are disabled.

But I'm unsure how to blackhole methods that use asserts...

Here is what I've tried so far:

@State(Scope.Benchmark)
@CompilerControl(CompilerControl.Mode.DONT_INLINE)
public class GcTest
{
    private String name = "actual";
    private Map<Integer, Integer> value;

    public GcTest()
    {
        value = new HashMap<>(5, 1f);
        for (int i = 0; i < 5; ++i)
            value.put(i, 5 - i);
    }

    @Benchmark
    @Fork(jvmArgsPrepend = "-da")
    @SuppressWarnings("EmptyMethod")
    public void emptyMethod()
    {
    }

    @Benchmark
    public void requireThat(Blackhole bh)
    {
        bh.consume(DefaultJavaValidators.requireThat(name, value).size().isGreaterThan(3));
    }

    @Benchmark
    @Fork(jvmArgsPrepend = "-da")
    public void assertThatWithAssertionsDisabled(Blackhole bh)
    {
        assert blackholeAssert(bh, DefaultJavaValidators.requireThat(name, value).size().isGreaterThan(3).
            elseThrowAssert());
    }

    private boolean blackholeAssert(Blackhole bh, boolean resultOfAssertion)
    {
        bh.consume(resultOfAssertion);
        return true;
    }


    @Benchmark
    @Fork(jvmArgsPrepend = "-ea")
    public void assertThatWithAssertionsEnabled(Blackhole bh)
    {
        assert blackholeAssert(bh, DefaultJavaValidators.requireThat(name, value).size().isGreaterThan(3).
            elseThrowAssert());
    }
}

Does this look correct?


Solution

  • Answering my own question:

    The trick is to define this method:

    private boolean blackholeAssert(Blackhole bh, boolean resultOfAssertion)
    {
        bh.consume(resultOfAssertion);
        return true;
    }
    

    and invoke it from inside the assert statement:

    @Benchmark
    @Fork(jvmArgsPrepend = "-da")
    public void assertThatWithAssertionsDisabled(Blackhole bh)
    {
        assert blackholeAssert(bh, DefaultJavaValidators.requireThat(name, value).size().isGreaterThan(3).
            elseThrowAssert());
    }