amazon-web-servicesaws-glueaws-glue-spark

What is the Scala and Java version for AWS Glue ETL job?

So far I'm using scala 2.11 with Java 8 to build the library used by the Glue ETL job. We're planning to upgrade to Scala 2.12 with Java 11 but not sure if they are supported by the Glue ETL.

Solution

  • The glue versions are listed here. The latest version supports Spark 2.4.3.

    In Spark 2.4.3, the default version of Scala is 2.11.