google-cloud-platformgoogle-cloud-storagegsutil

GCP: can we use output of one GSUTIL command embedded in other GSUTIL command

I have a requirement where have a file in GCS and I need to remove the header and trailer of a file and load the data into other GCS bucket.

I have tried using something like this, but its not working.

gsutil cp `gsutil cat gs://<bucket>/<object> | sed '1,1d; $d'` gs://<target_bucket>/<target_object>

Can someone share your thoughts, if you have any other better way to achieve this?

Solution

  • Use this format (the - as a file stream source)

    gcloud storage cat gs://<bucket>/<object> | sed '1,1d; $d' | gcloud storage cp - gs://<target_bucket>/<target_object>