I need to encrypt some columns in Hive tables. Normally I used for that aes_encrypt UDF function in Hive. However, seems like it doesn't exists in Spark.
Is there any existing alternative in Spark or I just need to write my own UDF?
There is an old ticket: https://issues.apache.org/jira/browse/SPARK-12567
However, it's not under active development.
Now, you have to write your own UDF.
You may also be interested in using Hive UDFs in Spark, then you have to use session with Hive support. More details here: https://github.com/bmc/spark-hive-udf