pythonpalantir-foundry

Can I export .csv (in Gzip) from Palantir but only the first part has headers?


I am having trouble with exporting .csv from Palantir. When I export all parts of zip it makes headers in every part. I could make some kind of powershell script to delete the headers but if there is other way directly in Palantir, it would help me a lot. I use python inside Palantir.

I know that you can export without headers, but that is only for all of the files. I didnt find a way to ignore all apart from the first part.


Solution

  • Is your data of the form/size where it could be coalesced into a single file? I'm not aware of how the gzip might affect this, but this has worked for me in the past when I needed to download a csv from Foundry.

    from transforms.api import transform, Input, Output
    
    @transform(
      output=Output("/path/to/python_csv"),
      my_input=Input("/path/to/input")
    )
    def my_compute_function(output, my_input):
     output.write_dataframe(my_input.dataframe().coalesce(1), output_format="csv")