python-3.xcurldownloadgzipgunzip

Curl to Python to download a GZ file?

curl --location --request GET 'https://sampleurl/sample.log.gz' \
--header 'Authorization: Bearer XXXTokenXXX' \
--data-raw '{
    "enabled": true
}' | gunzip -c

My requirement is to download a gz log file. The above is the sample curl which works as expected. How can I add this to python code?

Solution

  • Edit: Turns out there's a library dedicated to this exact problem. Here's the gzip library, and a stackoverflow post about this exact topic.

    Old answer: You may be able to do this using the requests library by passing in custom headers as a dictionary, and then writing the content of your response as a file.

    import requests
url = 'https://sampleurl/sample.log.gz'
headers = {'Authorization': 'Bearer XXXTokenXXX'}
response = requests.get(url, headers=headers)  # if the URL will redirect, include allow_redirects=True

with open('C:\\path\\to\\save\\to', 'wb') as file:
    file.write(response.content)

    You'll probably need to tinker around with that to get it working for your use-case (especially if that --data-raw bit is important), but that's the general idea. If you need to download a particularly large file or want to see some other alternative solutions, you can check out this older question.