NegativeArraySizeException error is raised when trying to create an edge between two vertexes in remote session

I'm trying to add an edge between two vertexes using the gremlin scala framework connected to a remote JanusGraph server. While this edge is created, I still get a "org.apache.tinkerpop.shaded.kryo.KryoException: java.lang.NegativeArraySizeException" error exception

The edge and vertexes do get created, but the error is still thrown and I can not catch it. I'm using JanusGraph 0.3.1, and tried with different versions of scala gremlin (3.3, 3.4), all leading to the same error.

val serializer = new GryoMessageSerializerV3d0(GryoMapper.build.addRegistry(TinkerIoRegistryV3d0.instance))
val cluster = Cluster.build.addContactPoint("localhost").port(8182).serializer(serializer).create
implicit val graph: ScalaGraph = EmptyGraph.instance.asScala.configure(_.withRemote(DriverRemoteConnection.using(cluster)))

val Founded = Key[String]("founded")
val Distance = Key[Int]("distance")

// create labelled vertex
val paris = graph + "Paris"

// create vertex with typed properties
val london = graph + ("London", Founded -> "43 AD")

// create labelled edges
paris --- ("OneWayRoad",  Distance -> 495) --> london
cluster.close()

Error message thrown at runtime

15:34:02.704 [gremlin-driver-loop-1] WARN  o.a.t.g.driver.MessageSerializer - Response [PooledUnsafeDirectByteBuf(ridx: 92, widx: 92, cap: 92)] could not be deserialized by org.apache.tinkerpop.gremlin.driver.ser.AbstractGryoMessageSerializerV3d0.
org.apache.tinkerpop.shaded.kryo.KryoException: java.lang.NegativeArraySizeException
Serialization trace:
id (org.apache.tinkerpop.gremlin.structure.util.reference.ReferenceEdge)
    at org.apache.tinkerpop.shaded.kryo.serializers.ObjectField.read(ObjectField.java:144)
    at org.apache.tinkerpop.shaded.kryo.serializers.FieldSerializer.read(FieldSerializer.java:557)
...
Caused by: java.lang.NegativeArraySizeException: null
    at org.apache.tinkerpop.shaded.kryo.io.Input.readBytes(Input.java:325)
[...]
15:34:02.705 [gremlin-driver-loop-1] ERROR o.a.t.g.d.Handler$GremlinResponseHandler - Could not process the response
io.netty.handler.codec.DecoderException: org.apache.tinkerpop.gremlin.driver.ser.SerializationException: org.apache.tinkerpop.shaded.kryo.KryoException: java.lang.NegativeArraySizeException
Serialization trace:
id (org.apache.tinkerpop.gremlin.structure.util.reference.ReferenceEdge)
    at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:98)
[...]
Caused by: org.apache.tinkerpop.gremlin.driver.ser.SerializationException: org.apache.tinkerpop.shaded.kryo.KryoException: java.lang.NegativeArraySizeException
Serialization trace:
id (org.apache.tinkerpop.gremlin.structure.util.reference.ReferenceEdge)
    at org.apache.tinkerpop.gremlin.driver.ser.AbstractGryoMessageSerializerV3d0.deserializeResponse(AbstractGryoMessageSerializerV3d0.java:159)
[...]
Caused by: org.apache.tinkerpop.shaded.kryo.KryoException: java.lang.NegativeArraySizeException
Serialization trace:
id (org.apache.tinkerpop.gremlin.structure.util.reference.ReferenceEdge)
    at org.apache.tinkerpop.shaded.kryo.serializers.ObjectField.read(ObjectField.java:144)
[...]
Caused by: java.lang.NegativeArraySizeException: null
    at org.apache.tinkerpop.shaded.kryo.io.Input.readBytes(Input.java:325)
[...]

The debugger shows me that the error is thrown when the edge is created. Using

val edge = g.V(paris).as("a").V(london).addE("test").iterate()

leads to the same error. Here's my gremlin-server.yaml configuraiton file

host: 0.0.0.0
port: 8182
scriptEvaluationTimeout: 180000
channelizer: org.apache.tinkerpop.gremlin.server.channel.WebSocketChannelizer
graphs: {
  graph: conf/gremlin-server/janusgraph-cql-es-server.properties,
  ConfigurationManagementGraph: conf/janusgraph-cql-configurationgraph.properties
}
scriptEngines: {
  gremlin-groovy: {
    plugins: { org.janusgraph.graphdb.tinkerpop.plugin.JanusGraphGremlinPlugin: {},
               org.apache.tinkerpop.gremlin.server.jsr223.GremlinServerGremlinPlugin: {},
               org.apache.tinkerpop.gremlin.tinkergraph.jsr223.TinkerGraphGremlinPlugin: {},
               org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin: {classImports: [java.lang.Math], methodImports: [java.lang.Math#*]},
               org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin: {files: [scripts/empty-sample.groovy]}}}}
serializers:
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { serializeResultToString: true }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
  # Older serialization versions for backwards compatibility:
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0, config: {ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { serializeResultToString: true }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}
processors:
  - { className: org.apache.tinkerpop.gremlin.server.op.session.SessionOpProcessor, config: { sessionTimeout: 28800000 }}
  - { className: org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor, config: { cacheExpirationTime: 600000, cacheMaxSize: 1000 }}
metrics: {
  consoleReporter: {enabled: true, interval: 180000},
  csvReporter: {enabled: true, interval: 180000, fileName: /tmp/gremlin-server-metrics.csv},
  jmxReporter: {enabled: true},
  slf4jReporter: {enabled: true, interval: 180000},
  gangliaReporter: {enabled: false, interval: 180000, addressingMode: MULTICAST},
  graphiteReporter: {enabled: false, interval: 180000}}
maxInitialLineLength: 4096
maxHeaderSize: 8192
maxChunkSize: 8192
maxContentLength: 65536
maxAccumulationBufferComponents: 1024
resultIterationBatchSize: 64
writeBufferLowWaterMark: 32768
writeBufferHighWaterMark: 65536

The error does not appears when working without using a remote server:

implicit val graph: ScalaGraph = EmptyGraph.instance

is working fine.

Solution

This issue typically points to a compatibility problem with Gryo which is usually exposed when TinkerPop versions are mixed. For the most part, Gryo tends to be backward compatible across versions, thus Gryo 1.0 from 3.3.3 will work with 3.3.4, but there are occasions where that is not always true (e.g. a bug is discovered in the core of the format and a breaking change is necessary.)

TinkerPop recommends that when using Gryo, that you align the TinkerPop version on the server with the client. So JanusGraph 0.3.1, uses TinkerPop 3.3.3, therefore your Gremlin Scala version should be 3.3.3.x (I'm pretty sure that Gremlin Scala binds their first three version numbers to TinkerPop's). It seems that you've tried that already, so let's next consider your configuration.

I note that you've added the TinkerIoRegistryV3d0 but since you're using JanusGraph, you might also need to add their custom IoRegistry:

GryoMessageSerializerV3d0(GryoMapper.build.addRegistry(JanusGraphIoRegistry.getInstance()))

You could add the TinkerIoRegistryV3d0 if your use case requires it - typically only useful for returning subgraphs. If none of that fixes the problem then my only suggestion would be to simplify heavily: remove all serializer configurations from Gremlin Server configuration except for the one you are using, make sure you can connect to that with just some simple scripts configuring your driver using just Gremlin Console and take note of what that configuration is to make the connection work so that you can port the configuration to Gremlin Scala.

I see that you have currently isolated the problem to:

val edge = g.V(paris).as("a").V(london).addE("test").iterate()

Note that this code doesn't do exactly what I think you want for a couple of reasons:

If you want the edge back you need to next() and not iterate()
That isn't adding an edge between "paris" and "london" - it's adding a self-referencing edge to "london". You need to specify the from() or to() after addE().

I hope something in there helps.