I am having trouble with exporting .csv from Palantir. When I export all parts of zip it makes headers in every part. I could make some kind of powershell script to delete the headers but if there is other way directly in Palantir, it would help me a lot. I use python inside Palantir.
I know that you can export without headers, but that is only for all of the files. I didnt find a way to ignore all apart from the first part.
Is your data of the form/size where it could be coalesced into a single file? I'm not aware of how the gzip might affect this, but this has worked for me in the past when I needed to download a csv from Foundry.
from transforms.api import transform, Input, Output
@transform(
output=Output("/path/to/python_csv"),
my_input=Input("/path/to/input")
)
def my_compute_function(output, my_input):
output.write_dataframe(my_input.dataframe().coalesce(1), output_format="csv")