I'm currently conducting a research project comparing the performance of Ballerina programming language with Spring Boot. I'm using JMeter tests to analyze various performance metrics.
One concern I have is regarding the consistency of the results across different environments. Specifically, I want to know if the performance metrics I'm obtaining on my local machine would be similar if I were to run the same tests on another computer. If not, what are the potential factors that could lead to discrepancies in the results?
Should I expect similar performance results across different machines with the same configurations, or would it be more reliable to conduct the tests in a cloud environment? What are the advantages and disadvantages of each approach in terms of ensuring consistent and accurate performance measurements?
For reference, you can find the results of my performance tests in my GitHub repository here. Any insights or experiences with conducting performance tests for Ballerina and Spring Boot in different environments would be greatly appreciated. Thank you in advance for your help!
In my research comparing the performance of Ballerina and Spring Boot, I've conducted JMeter tests on both platforms to measure various performance metrics such as response time, throughput, and resource utilization. I've have executed these tests on my local machine and now i wonder if i test it on another computer i would get a similar result.
I went through your findings so far, and they look great.
Should I Expect Consistency Across Different Environments?
I don't think you can expect consistency across different environments. There are several factors to consider, such as OS, JVM, Garbage collector, etc. What we've done in the past is to pick a certain environment (a Cloud environment is preferred) and document the details for someone who wants to replicate your tests.
I can see that you've compared Ballerina(AOT), Ballerina, and Sprint Boot. My recommendation is to add Spring Boot(AOT), if available, to this mix. That was the first thing I looked for when I saw Ballerina(AOT)