I'm using a scylla cluster with 9 nodes and replication factor to be 3. One application opens one connections to it, reading and writing data.
However, I notice in the scylla CQL optimization monitoring dashboard, the percentage for cross shard is very high, around 150%.
Is that normal or how can I optimize it.
First, there is an issue with this metric, and it was removed in the latest dashboard https://github.com/scylladb/scylla-monitoring/issues/659
Second, the way to reduce cross-shard messages is using Token Aware drivers, if you can. There are two Token Aware drivers at the moment: Go and Java https://docs.scylladb.com/using-scylla/scylla_drivers/