[SOLVED] scala type mismatch osmosis

scala type mismatch osmosis

I try to get the example from this open book about map matching to work. I am using scala 2.10 and spark 2.0.2.

Unfortunatelly the syntax and functions have changed.

case class NodeEntry(nodeId: Long, latitude: Double, longitude: Double, tags: Array[String])

val nodeDS = nodes.map{node => 
NodeEntry(node.getId, 
   node.getLatitude, 
   node.getLongitude, 
   node.getTags.map(_.getValue).toArray
)}.toDS.cache

I get the Error "value map is not a member of java.util.List[org.openstreetmap.osmosis.core.domain.v0_6.WayNode]"

and "value toDS is not a member of scala.collection.mutable.ArrayBuffer[Nothing] possible cause: maybe a semicolon is missing before `value toDS'?"

I tried to change the nodes.map to the following:

val nodeDS = nodes.map { node => 
  NodeEntry(node.getId, 
      node.getLatitude, 
      node.getLongitude, 
      node.getTags.toArray() 
  )}

But then I get this Error: type mismatch; found : Array[Object] required: Array[String] Note: Object >: String, but class Array is invariant in type T. You may wish to investigate a wildcard type such as _ >: String. (SLS 3.2.10)

Solution

map is not available on Java collections inorder to make it available import JavaConversions

import scala.collection.JavaConversions._

Example

Scala REPL

scala> val l = new java.util.ArrayList[String]()
l: java.util.ArrayList[String] = []

scala> l.add("scala")
res0: Boolean = true

scala> l.add("haskell")
res1: Boolean = true

scala> l.map(x => x.reverse)
<console>:13: error: value map is not a member of java.util.ArrayList[String]
       l.map(x => x.reverse)
     ^

scala> import scala.collection.JavaConversions._
import scala.collection.JavaConversions._

scala> l.map(x => x.reverse)
res3: scala.collection.mutable.Buffer[String] = ArrayBuffer(alacs, lleksah)

Notice that after importing map is now available on the java ArrayList