snowflake-cloud-data-platform

Offloading empty Snowflake table into stage


I have a requirement to export rows from Snowflake table into a file in an internal Snowflake stage. If there are zero rows, I still want the file to be exported with just columns and no data. My code is Python/Snowpark so I probably can write a workaround but was hoping there is solution for this problem in Snowflake. I could not find anything here: https://docs.snowflake.com/en/sql-reference/sql/copy-into-location#label-copy-into-location-copyoptions

code I am using:

        df.write.copy_into_location(stage_name + file_name,
                          file_format_type = 'csv',
                          header = True, 
                          overwrite = True ,
                          single = True,
                        format_type_options={"COMPRESSION": "NONE", "NULL_IF": (),"RECORD_DELIMITER":"\r\n", "FIELD_OPTIONALLY_ENCLOSED_BY":'"'},
                    )

Solution

  • As Lukasz Szozda already mentioned, Snowflake's COPY command does not unload data if the souce has 0 rows, and df.write.copy_into_location method uses COPY command to unload the data.

    To overcome this, you may do a trick and write the column names as data:

        if ( df.count() == 0):
    
            header_values = []
            for field in df.schema.fields:
                header_values.append( field.name ) 
            df = session.createDataFrame( [header_values] )            
    
            df.write.copy_into_location(stage_name + file_name,
                          file_format_type = 'csv',
                          header = False, 
                          overwrite = True ,
                          single = True,
                        format_type_options={"COMPRESSION": "NONE", "NULL_IF": (),"RECORD_DELIMITER":"\r\n", "FIELD_OPTIONALLY_ENCLOSED_BY":'"'},
                    )
        else:
            df.write.copy_into_location(stage_name + file_name,
                          file_format_type = 'csv',
                          header = True, 
                          overwrite = True ,
                          single = True,
                        format_type_options={"COMPRESSION": "NONE", "NULL_IF": (),"RECORD_DELIMITER":"\r\n", "FIELD_OPTIONALLY_ENCLOSED_BY":'"'},
                    )