[SOLVED] How to raise Exception in OPENSOCKETFUNCTION of pycurl

How to raise Exception in OPENSOCKETFUNCTION of pycurl

As per https://curl.haxx.se/libcurl/c/CURLOPT_OPENSOCKETFUNCTION.html, libcurl document said that this callback can be used for IP address blacklisting, so I decided to use it to block all private IP address.

I have tried raise Exception directly in my custom callback function, but it won't propagate any exception to curl.perform except timeout occurred.

Is it possible to catch the exception I raised in callback function from curl.perform() ?

Thanks in advance.

Solution

It may be possible for pycurl to propagate Python exceptions in callbacks out of perform calls, however this is not currently implemented. Right now each callback should return an appropriate value to indicate the error; in case of opensocket callback, this value is CURL_SOCKET_BAD. (Unfortunately, pycurl does currently not expose this constant either; use -1 instead.)

One way to provide rich error information is to store it on the Curl object itself, as follows:

import pycurl, random, socket

class ConnectionRejected(Exception): pass

def opensocket(curl, purpose, curl_address):
    if random.random() < 0.5:
        curl.exception = ConnectionRejected('Rejecting connection attempt in opensocket callback')
        # should be pycurl.SOCKET_BAD in the next release
        return -1

    family, socktype, protocol, address = curl_address
    s = socket.socket(family, socktype, protocol)
    s.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
    return s

c = pycurl.Curl()
c.setopt(c.URL, 'http://pycurl.io')
c.exception = None
c.setopt(c.OPENSOCKETFUNCTION,
    lambda purpose, address: opensocket(c, purpose, address))
try:
    c.perform()
except pycurl.error as e:
    if e.args[0] == pycurl.E_COULDNT_CONNECT and c.exception:
        print(c.exception)
    else:
        print(e)

Additionally pycurl doesn't currently handle -1/CURL_SOCKET_BAD return quite correctly - it complains that -1 is not a valid return value (but functionality is not otherwise affected). https://github.com/pycurl/pycurl/pull/484 will repair this.