restservicebenchmarkingcaliper

Using Caliper to benchmark REST Services ... or not?


I want to compare the response time between different services I made. I heard about Caliper so I decided to use it in a very simple way to make calls and measure them, with some functions like that:

public void timeCallingService(int reps) {
    String url = "http://localhost:8080/myservice/rest/"+size;
    for (int i = 0; i < reps; i++) {
        new Client(url);
    }
}

where size is a given parameter for Caliper and Client a class I wrote which just make a call to the service and check if the status answer is ok.

Everything's works fine except for some services which take too much time to answer, so I have an error in Caliper :

Error: Runtime 1.06027641E11ns/rep out of range 0.1-1.0E10

I tried to play with some parameters like warmupMillis and runMillis but it doesn't seem to work.

So :

  1. is there any easy way to solve my problem?
  2. if there isn't, which benchmark framework can I use to do my comparison, given the fact I don't want anything complicated?

Solution

  • I apologize for that. Caliper 0.5 has some quirks like this, but Caliper 1.0 which we're working (a) is less anal-retentive about what it expects to see when timing like this, (b) makes a lot of values configurable (such as the timing interval; you can make it longer than 1 second), and (c) also lets you plug in a different measuring instrument if the MicrobenchmarkInstrument's logic just doesn't work for you.

    Unfortunately I can't promise a release date yet. :(