How to get Benchmark.js to do setup/teardown each time my benchmarked code is run (not just each cycle)?

I am trying to benchmark an Object's member function using Benchmark.js. Testing the function is made difficult by several factors:

• Creation of the object is asynchronous (I could mock that part)
• The member function is expensive
• The member function is smart enough to only run once

Let's say it looks like this:

class Something {

  constructor(){
    // async ops
    this.expensiveValue = null;
  }

  expensiveOperation () {

    if (this.expensiveValue === null) {
      // Do expensive operation
      this.expensiveValue = result; // a non-null value
    }

  }

}

Now, I want to benchmark expensiveOperation. But due to its limitations, I also need to "reset" the object each run.

As far as I can tell,benchmark doesn't support per-run setups. I feel like making the reset part of the run isn't the best practice either, because it pollutes what I'm actually trying to benchmark.

I've looked at Benchmark.setup, but that only executes per-cycle, not per-run.

Am I missing something? Is there another benchmark option I can use? Or am I approaching this incorrectly?

🎶 You can't always get what you want... ♩

Benchmark.js does not perform setup/teardown per iteration of your function, but only for each benchmark.js cycle, which is usually whatever it can run in 5 seconds, which might be hundreds to thousands to millions of calls. It does this for very good reason, as explained by the authors of Benchmark.js in Bulletproof JavaScript benchmarks.

If you believe those reasons don't apply to you, then I'd ask why you are even using Benchmark.js. You could easily write a loop and measure the duration of each call and take the average yourself, in a handful of lines of code. You don't need a fancy library.

You can't always get what you want

And if you try sometime, you might find

You get ♫   what you need.   ♬♪

To be honest, yes, you are approaching this incorrectly. Your expensiveOperation () is designed to be efficient for all subsequent calls, so of course a good benchmark should reflect that. The cost of the first call is amortized over all subsequent calls. Benchmark.js will try to measure the efficiency of your method as designed. That's the point.

Think about your underlying goal, and why it is you want to reset for each iteration. You don't want to benchmark expensiveOperation (), but only this part of the method:

      // Do expensive operation
      this.expensiveValue = result; // a non-null value

So simply factor that out into a method or function, and benchmark that. :)