scalaapache-sparkuser-defined-functionsudf

Define return value in Spark Scala UDF

Imagine the following code:

def myUdf(arg: Int) = udf((vector: MyData) => {
  // complex logic that returns a Double
})

How can I define the return type for myUdf so that people looking at the code will know immediately that it returns a Double?

Solution

  • Spark functions define several udf methods that have the following modifier/type: static <RT,A1, ..., A10> UserDefinedFunction

    You can specify the input/output data types in square brackets as follows:

    def myUdf(arg: Int) = udf[Double, MyData]((vector: MyData) => {
  // complex logic that returns a Double
})