Paramiko request_port_forward with and without handler

TLDR:

can't read data from incoming paramiko port forwarding connections when using callbacks. See this gist for a demonstration:
https://gist.github.com/tatome/0d3e6479f35b25bbb31c9f94610eab6b

The gist also demonstrates the solution to the problem.

Detailed Question:

There's an example script that demonstrates how to use paramiko to set up a reverse tunnel here.

The script requests port forwarding and then keeps accepting connections in the main thread, and delegates handling these connections to another thread:

transport.request_port_forward("", server_port)
while True:
    chan = transport.accept(1000)
    if chan is None:
        continue
    thr = threading.Thread(
        target=handler, args=(chan, remote_host, remote_port)
    )

Unfortunately, with this method, we don't know which port an incoming connection accessed, so we can't have multiple tunnels on different server ports that we treat differently.

However, we can pass a handler argument to request_port_forward, and handler does get the host name and port that the client accessed:

def handler(channel, source_address, target_address):
    target_host, target_port = target_address
    ...

client.get_transport().request_port_forward('', port, handler)

Here's the problem: For some reason, the channel retrieved via transport.accept() and the channel passed to handler behave differently for me.

When I do this:

client.connect(
    'localhost',
    22,
    username=USER,
    look_for_keys=True
)

def handler(channel, source_address, target_address):
    print("Selecting...")
    selects = select.select([channel],[],[])
    print("Selected: " + str(selects))

transport = client.get_transport()
p = transport.request_port_forward('', 30000)
print(p)
channel = transport.accept(1000)
handler(channel, None, None)

and then, on a shell:

wget localhost:30000

I get this:

30000
Selecting...
Selected: ([<paramiko.Channel 0 (open) window=2097152 in-buffer=130 -> <paramiko.Transport at 0x43b80fd0 (cipher aes128-ctr, 128 bits) (active; 1 open channel(s))>>], [], [])

If I pass the handler to request_port_forward, however, like this:

def handler(channel, source_address, target_address):
    print("Selecting...")
    selects = select.select([channel],[],[])
    print("Selected: " + str(selects))

transport = client.get_transport()
transport.request_port_forward('', 30000, handler)

I get this:

Selecting...

and when I debug the script it will stop in the line that says selects = select.select([channel],[],[]).

wget says it sent the HTTP request, but my script never reads anything from the connection.

Questions:

• What am I doing wrong?
• Why does select.select([channel]) does something else, depending on whether it's returned by transport.accept() or passed to the handler?
• How can you use the same transport for multiple different reverse tunnels?

I'd expect this to be some threading issue. The difference between the two codes is that in the first, the handler is called on the main thread, while in the second, the handler is called on another thread. I do not think that Paramiko is thread safe.

What I would try is to use handler only to notify the main thread loop (after request_port_forward) that a tunnel was opened and let it process it.

Quick and dirty code (just to test if I'm right or not)

tunnel_channel = None

def handler(channel, source_address, target_address):
    global tunnel_channel
    print("Tunnel opened.")
    tunnel_channel = channel

transport = client.get_transport()
transport.request_port_forward('', 30000, handler)

while True:
    if tunnel_channel != None:
        channel = tunnel_channel
        tunnel_channel = None
        print("Selecting...")
        selects = select.select([channel],[],[])
        print("Selected: " + str(selects))

    time.sleep(1)