We have a basic 2 node Cassandra cluster. Both nodes run version 3.9 with minimal configurations to enable clustering. One of the nodes is sending wrong generation which causes the other node to show the warning
WARN [GossipStage:1] Gossiper.java:1146 - received an invalid gossip generation for peer /10.3.185.234; local time = 1479469393, received generation = 1872927836
Node-1 which is causing the issue has this output from
nodetool gossipinfo
/10.3.185.234
generation: 1872927836
1872927836 epoch is a far away date (Tue, 08 May 2029 09:43:56 GMT). Node 2 is legitimately discarding the packet. Can I somehow fix node-1 to send the correct generation?
Fixed the issue by changing the gossip_generation value in system.local table by using cqlsh
update system.local set gossip_generation = 1479472176 where key='local';
Restart the service after this change