We are upgrading our SonarQube from 4.5.X to 5.6.X, compute engine is a new concept and on the paper is pretty shinny. But when put to work, it does not perform much! help me get through if you had made it
I choose a test projects that was scanned under 4 minutes with SonarQube 4.5.4. Now the initial scan taking place with (gradle) scanner takes 3 minutes. And CE background tasks runs forever, out of the test projects, first one completed in 183 minutes, other in 263 minutes, third one is still running.
Obviously I got digging and saw this stackoverflow question - Sonarqube background tasks running forever and increased the memory as quoted as answer, but its is not working for me.
sonar.ce.javaOpts=-Xmx4096m -Xms512m -XX:+HeapDumpOnOutOfMemoryError -Djava.net.preferIPv4Stack=true
sonar.ce.workerCount=5
Though worker count is 5, I'm not running parallel tasks now. From the completed job's logs I'm able to see some alarming time info!
2018.02.13 16:56:07 INFO [o.s.s.c.s.ComputationStepExecutor] Compute complexity measures | time=335ms
2018.02.13 16:56:07 INFO [o.s.s.c.s.ComputationStepExecutor] Load measure computers | time=6ms
2018.02.13 21:18:34 INFO [o.s.s.c.s.ExecuteVisitorsStep] Execution time for each component visitor:
2018.02.13 21:18:34 INFO [o.s.s.c.s.ExecuteVisitorsStep] - LoadComponentUuidsHavingOpenIssuesVisitor | time=2483ms
2018.02.13 21:18:34 INFO [o.s.s.c.s.ExecuteVisitorsStep] - IntegrateIssuesVisitor | time=15743879ms
2018.02.13 21:18:34 INFO [o.s.s.c.s.ExecuteVisitorsStep] - CloseIssuesOnRemovedComponentsVisitor | time=0ms
2018.02.13 21:18:34 INFO [o.s.s.c.s.ExecuteVisitorsStep] - QualityModelMeasuresVisitor | time=143ms
2018.02.13 21:18:34 INFO [o.s.s.c.s.ExecuteVisitorsStep] - NewQualityModelMeasuresVisitor | time=73ms
2018.02.13 21:18:34 INFO [o.s.s.c.s.ExecuteVisitorsStep] - LastCommitVisitor | time=4ms
2018.02.13 21:18:34 INFO [o.s.s.c.s.ExecuteVisitorsStep] - MeasureComputersVisitor | time=25ms
2018.02.13 21:18:34 INFO [o.s.s.c.s.ComputationStepExecutor] Execute component visitors | time=15746700ms
2018.02.13 21:18:37 INFO [o.s.s.c.s.ComputationStepExecutor] Compute measure variations | time=3290ms
2018.02.13 21:18:37 INFO [o.s.s.c.s.ComputationStepExecutor] Compute Quality Gate measures | time=54ms
2018.02.13 21:18:37 INFO [o.s.s.c.s.ComputationStepExecutor] Compute Quality profile measures | time=25ms
2018.02.13 21:18:38 INFO [o.s.s.c.s.ComputationStepExecutor] Generate Quality profile events | time=33ms
2018.02.13 21:18:38 INFO [o.s.s.c.s.ComputationStepExecutor] Generate Quality gate events | time=6ms
Execute component visitor, particularly IntegrateIssuesVisitor takes hell a lot of time, 262 minutes out of total 263 minutes required for execution. How to avoid this?
I don't see any cross project analysis disable option, is it related to that?
Additional information: I use Oracle DB
[Update] Crossed out curse on CE. My bad, CE is awesome! Now some projects are analysed and published in 3 seconds!!
Solved!
It had to do with Database, as discussions on SonarQube Google groups pointed out.
From https://docs.sonarqube.org/display/PLUG/Managing+Compute+Engine+Performance, It is clear that Compute Engine performance are tied to,
Our memory consumption and CPU usage were always minimal and so straight away we ruled out last two. But we didn't know what was wrong yet, if anything CE logs are useless in every way since log level is not applied to CE logs, you are stuff with only INFO no DEBUG or TRACE levels. Still we took a hint,
INFO [o.s.s.c.s.ExecuteVisitorsStep] Execution time for each component visitor:
INFO [o.s.s.c.s.ExecuteVisitorsStep] - LoadComponentUuidsHavingOpenIssuesVisitor | time=58ms
INFO [o.s.s.c.s.ExecuteVisitorsStep] - IntegrateIssuesVisitor | time=41147606ms
INFO [o.s.s.c.s.ExecuteVisitorsStep] - CloseIssuesOnRemovedComponentsVisitor | time=1ms
INFO [o.s.s.c.s.ExecuteVisitorsStep] - QualityModelMeasuresVisitor | time=206ms
INFO [o.s.s.c.s.ExecuteVisitorsStep] - NewQualityModelMeasuresVisitor | time=119ms
INFO [o.s.s.c.s.ExecuteVisitorsStep] - LastCommitVisitor | time=6ms
INFO [o.s.s.c.s.ExecuteVisitorsStep] - MeasureComputersVisitor | time=29336ms
IntegrateIssuesVisitor was taking outrageous amount of time to execute, the log was only that informative and the why part was still mystery. So we decided to dig the code, thank god SonarQube is open source but soon enough or DBA team rang us up why our query is behaving so badly. And it stuck us, indexes we Analyzed the transaction tables of Sonar and we are good now.
tl;dr : Keep your table statistics updated so optimization are always working in your favor. Analyze tables.