Neither the developer nor the API documentation includes any reference about what options can be passed in DataFrame.saveAsTable
or DataFrameWriter.options
and they would affect the saving of a Hive table.
My hope is that in the answers to this question we can aggregate information that would be helpful to Spark developers who want more control over how Spark saves tables and, perhaps, provide a foundation for improving Spark's documentation.
The reason you don't see options
documented anywhere is that they are format-specific and developers can keep creating custom write formats with a new set of options
.
However, for few supported formats I have listed the options as mentioned in the spark code itself: