I am working on Sonarqube 5.1. It used to work fine earlier, later i got this no space left on device error.So i have deleted some unwanted files and restarted the service. Now i can see that the service status is running, but i cannot access the webpage(since sonar webservice is not started) I have checked the logs and i find the below error.
2015.08.03 10:58:56 INFO es[o.elasticsearch.node] [sonar-1438592314778] started
2015.08.03 10:59:03 INFO es[o.e.gateway] [sonar-1438592314778] recovered [6] indices into cluster_state
2015.08.03 11:00:44 WARN es[o.e.indices.cluster] [sonar-1438592314778] [sourcelines][0] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: [sourcelines][0] failed to recover shard
at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:287) ~[elasticsearch-1.4.4.jar:na]
at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:132) ~[elasticsearch-1.4.4.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:114 2) [na:1.8.0_40]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_40]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40]
Caused by: org.elasticsearch.index.translog.TranslogCorruptedException: translog corruption while reading from stream
at org.elasticsearch.index.translog.ChecksummedTranslogStream.read(ChecksummedTranslogStream.java:70) ~[elasticsearch-1.4.4.jar:na]
at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:257) ~[elasticsearch-1.4.4.jar:na]
... 4 common frames omitted
Caused by: org.elasticsearch.ElasticsearchException: failed to read [sourceline][22c2d1d1-8e73-47c0-b9fa-f8f5cc96b93f_796]
at org.elasticsearch.index.translog.Translog$Index.readFrom(Translog.java:520) ~[elasticsearch-1.4.4.jar:na]
at org.elasticsearch.index.translog.ChecksummedTranslogStream.read(ChecksummedTranslogStream.java:68) ~[elasticsearch-1.4.4.jar:na]
... 5 common frames omitted
Caused by: org.elasticsearch.ElasticsearchIllegalArgumentException: No version type match [105]
at org.elasticsearch.index.VersionType.fromValue(VersionType.java:307) ~[elasticsearch-1.4.4.jar:na]
at org.elasticsearch.index.translog.Translog$Index.readFrom(Translog.java:517) ~[elasticsearch-1.4.4.jar:na]
... 6 common frames omitted
2015.08.03 11:00:44 WARN es[o.e.c.action.shard] [sonar-1438592314778] [sourcelines][0] sending failed shard for [sourcelines][0], node[5eRgYcVUTNCNLjhhzdiHGA], [P], s[INITIALIZING], indexUUID [JaH5lnRnRVOFD95Nw28W5Q], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[sourcelines][0] failed to recover shard]; nested: TranslogCorruptedException[translog corruption while reading from stream]; nested: ElasticsearchException[failed to read [sourceline][22c2d1d1-8e73-47c0-b9fa-f8f5cc96b93f_796]]; nested: ElasticsearchIllegalArgumentException[No version type match [105]]; ]]
2015.08.03 11:00:44 WARN es[o.e.c.action.shard] [sonar-1438592314778] [sourcelines][0] received shard failed for [sourcelines][0], node[5eRgYcVUTNCNLjhhzdiHGA], [P], s[INITIALIZING], indexUUID [JaH5lnRnRVOFD95Nw28W5Q], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[sourcelines][0] failed to recover shard]; nested: TranslogCorruptedException[translog corruption while reading from stream]; nested: ElasticsearchException[failed to read [sourceline][22c2d1d1-8e73-47c0-b9fa-f8f5cc96b93f_796]]; nested: ElasticsearchIllegalArgumentException[No version type match [105]]; ]]
do i have to delete shards? if yes how to do that? how to recover the shard? is indexing wrong? where do i find [sourcelines] on my machine ? help me with this.
Thankyou in advance.
Your ES index seems to be corrupted.
All base information is stored in the DB. Do not delete or update it manually.
On the contrary, if you delete the ES indexes, they will simply be rebuilt at the next startup (which may take some time depending on the quantity of data you have).
To achieve that, delete the es directory in the data directory of your SQ instance.