I've been vetting Static Analysis tools and I've recently come across both Atlassian Clover and SonarQube. The two products seem remarkably similar and virtually identical from a server perspective, yet I can't find a good comparison of them online.
I've also been vetting their IntelliJ plugins, and this where I see vast differences between the two. Clover's IDE integration is amazing, pointing out exact lines of code that aren't covered by unit tests. However, the SonarQube server has this functionality, but I'm not sure the $300/person cost of Clover is worth this IDE convenience.
Sonar's plugin simply seems to point out code issues in the IDE, which is good, but IntelliJ has functionality for this already. Also, does Clover lack this in their plugin, or do I simply not see it because I haven't put the Clover plugin in front of a server yet?
Lastly, I've also seen that Sonar can consume reports generated by Clover. Does anyone have any experience with this? Does the SonarQube server sufficiently replace the Clover server by utilizing these reports? If not, what does Sonar lack?
For reference: http://docs.codehaus.org/display/SONAR/Clover+Plugin
Some background: The product being analyzed is a Java web project being build with Maven. Both tools seem to have appropriate Maven integration.
Disclaimer: I'm a Clover developer at Atlassian.
But I'll try to be as objective as possible, of course.
Let me emphasize one difference between Clover and Sonar first:
Clover is mainly a code coverage tool. It tracks both total coverages as well as per-test coverage. It has some code metrics in addition to this, but it's not a typical static code analysis tool like Checkstyle, FindBugs or PMD.
Sonar (from my perspective) is mainly a data aggregation tool, which can collect various kinds of data (like code coverage, static analysis results, code metrics) from various tools and present them in one place.
What is similar in these two tools that both of them can create rich reports.
Having said this, let me answer your questions.
Clover's IDE integration is amazing, pointing out exact lines of code that aren't covered by unit tests [...] but I'm not sure the $300/person cost of Clover is worth this IDE convenience.
You have to answer it yourself :-) Few things worth consideration:
How does your developer run unit tests - do they run them in IDE before commit? Do you have a "green build" policy? If both answers are yes then having Clover in IDEA may be worth it.
How much time takes the execution of unit tests in IDE? How frequently they're launched? If they take a long time and are frequently launched, then using Clover's Test Optimization feature in IDEA could be interesting.
Do you have your tests split into several build plans running on CI server? Running in specific environment configurations? in such case, server reports could be more valuable than in IDE
do your developers prefer to see code coverage directly in their IDEs or rather clicking through the HTML report in a browser?
Do you expect to see any productivity boost of your team thanks to having source code with coverage highlighting in IDE? How much? Is the 'time saved * salary > clover license price'?
Sonar's plugin simply seems to point out code issues in the IDE, which is good, but IntelliJ has functionality for this already. Also, does Clover lack this in their plugin [...]?
Clover does not perform static analysis, and thus it does not show code bugs. Neither in its HTML report nor IDE plugins (IDEA/Eclipse).
Lastly, I've also seen that Sonar can consume reports generated by Clover. [...] Does the SonarQube server sufficiently replace the Clover server by utilizing these reports? If not, what does Sonar lack?
I'm not 100% sure (please correct me if I'm wrong) but I think the Sonar Clover Plugin parses Clover's XML report file (at least Clover plugins for Jenkins, Hudson and Bamboo work this way) to get some numbers to display. Which means that you won't see Clover's HTML report in Sonar with detailed source line coloring, per-test coverage, tag clouds, etc.
Cheers Marek