pythonapache-sparkpysparkapache-spark-sql

convert columns of pyspark data frame to lowercase

I have a dataframe in pyspark which has columns in uppercase like ID, COMPANY and so on

I want to make these column names to id company and so on. Bacially convert all the columns to lowercase or uppercase depending on the requirement.

I want to do in such away that the data types of the columns remain the same.

How can we do that?

Solution

  • Use columns field from DataFrame

    df = // load
for col in df.columns:
    df = df.withColumnRenamed(col, col.lower())

    Or, as @zero323 suggested:

    df.toDF(*[c.lower() for c in df.columns])