python-requests

how to construct the curl command from python requests module?


Python requests is a good module to ease my web REST API access programming, I usually do like below

import json
url = 'https://api.github.com/some/endpoint'
payload = {'some': 'data'}
headers = {'Content-type': 'application/json', 'Accept': 'application/json'}

r = requests.post(url, data=json.dumps(payload), headers=headers)

And when there is error comes out, I want to see what happen behind it. Constructing the curl command to reproduce in command line is the common way, since this is the standard way which is most described in RESP API document

try:
    r = requests.post(url, data=json.dumps(payload), headers=headers)
except Exception as ex:
    print "try to use curl command below to reproduce"
    print curl_request(url,"POST",headers,payload)

It will be nice I can generate curl command sample for this request, see good example in libcloud's debug, I can't find a simple way to construct, below are the method I want to create by myself.

# below code is just pseudo code, not correct 
def curl_request(url,method,headers,payloads):
    # construct curl sample from requests' structure
    # $ curl -v -H "Accept: application/json" -H "Content-type: application/json" 
    # -d '{"some":"data"}' 
    # -X POST https://api.github.com/some/endpoint
    request = "curl -v "
    for header in headers:
        print header
        request = request + '-H "' + header + ": " + headers[header] + '" '
    for payload in payloads:
        request = request + '-d {} "' + payload + ": " + payloads[payload] + '" '         
    request = request + "-X %s %s" % (method,url)
    return request

It will also be nice if we have method in requests already


Below are the final solution get the answer, works for me. Show it here for your reference

def curl_request(url,method,headers,payloads):
    # construct the curl command from request
    command = "curl -v -H {headers} {data} -X {method} {uri}"
    data = "" 
    if payloads:
        payload_list = ['"{0}":"{1}"'.format(k,v) for k,v in payloads.items()]
        data = " -d '{" + ", ".join(payload_list) + "}'"
    header_list = ['"{0}: {1}"'.format(k, v) for k, v in headers.items()]
    header = " -H ".join(header_list)
    print command.format(method=method, headers=header, data=data, uri=url)    

Solution

  • This method existed in requests once upon a time but it is far from being remotely relevant to the module. You could create a function that takes a response and inspects its request attribute.

    The request attribute is a PreparedRequest object so it has headers, and body attributes. The body is what you pass to curl with -d and the headers can be generated as you did above. Finally you'll want to pluck off the url attribute from the request object and send that. The hooks don't matter to you unless you're doing something with a custom authentication handler.

    req = response.request
    
    command = "curl -X {method} -H {headers} -d '{data}' '{uri}'"
    method = req.method
    uri = req.url
    data = req.body
    headers = ['"{0}: {1}"'.format(k, v) for k, v in req.headers.items()]
    headers = " -H ".join(headers)
    return command.format(method=method, headers=headers, data=data, uri=uri)
    

    That should work. Your data will be properly formatted whether it is as multipart/form-data or anything else.